Sep 15 12:02:27.054231 ip-10-0-130-139 systemd[1]: Starting Kubernetes Kubelet... Sep 15 12:02:27.389677 ip-10-0-130-139 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 12:02:27.389677 ip-10-0-130-139 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 15 12:02:27.389677 ip-10-0-130-139 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 12:02:27.389677 ip-10-0-130-139 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 15 12:02:27.389677 ip-10-0-130-139 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 12:02:27.391972 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.391871 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 15 12:02:27.395451 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395432 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 12:02:27.395451 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395449 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 12:02:27.395451 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395453 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395464 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395467 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395470 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395474 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395477 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395481 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395483 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395486 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395489 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395492 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395494 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395497 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395500 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395502 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395506 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395510 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395513 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395516 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 12:02:27.395564 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395519 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395521 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395523 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395526 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395530 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395534 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395536 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395538 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395541 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395544 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395546 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395549 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395551 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395554 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395557 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395560 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395564 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395566 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395569 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 12:02:27.396090 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395572 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395574 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395577 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395581 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395584 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395587 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395590 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395592 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395595 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395598 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395600 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395603 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395605 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395609 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395612 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395614 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395617 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395620 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395623 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395625 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 12:02:27.396685 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395629 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395631 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395633 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395636 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395638 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395641 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395643 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395645 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395648 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395651 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395653 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395656 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395660 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395662 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395665 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395667 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395670 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395672 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395675 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395677 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 12:02:27.397323 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395680 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395682 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395684 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395687 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395689 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395692 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395694 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395697 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395699 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395702 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395704 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395707 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395709 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395713 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395715 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395718 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395720 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395723 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.395725 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397809 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 12:02:27.397813 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397821 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397825 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397829 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397832 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397835 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397837 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397840 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397842 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397848 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397851 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397854 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397857 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397859 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397862 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397864 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397867 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397870 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397873 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397876 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397878 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 12:02:27.398319 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397884 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397886 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397890 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397894 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397897 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397899 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397903 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397905 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397926 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397932 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397935 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397938 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397941 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397946 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397950 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397953 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397956 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397959 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397962 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 12:02:27.398817 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397965 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397967 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397970 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397973 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397976 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397978 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397983 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397985 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397988 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397990 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397993 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397996 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.397998 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398001 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398004 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398006 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398009 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398011 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398031 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398036 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 12:02:27.399310 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398040 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398043 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398046 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398050 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398052 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398055 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398059 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398063 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398066 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398071 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398076 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398079 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398082 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398085 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398088 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398090 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398093 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398096 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398098 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 12:02:27.399819 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398101 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398105 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398108 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398114 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398117 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398120 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398122 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398125 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398127 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398130 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398132 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398135 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398137 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398140 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398143 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398146 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398151 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398153 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398155 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 12:02:27.400316 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398158 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398766 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398771 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398775 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398778 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398781 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398784 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398788 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398791 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398794 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398797 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398800 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398803 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398809 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398812 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398815 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398817 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398820 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398823 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 12:02:27.400776 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398825 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398828 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398831 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398834 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398836 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398839 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398844 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398847 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398850 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398853 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398855 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398857 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398860 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398863 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398868 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398871 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398873 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398876 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398881 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398883 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 12:02:27.401289 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398886 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398890 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398893 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398896 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398899 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398902 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398905 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398924 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398927 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398930 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398935 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398938 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398940 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398943 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398945 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398948 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398951 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398954 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398956 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 12:02:27.401788 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398959 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398961 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398965 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398967 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398972 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398975 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398978 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398981 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398984 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398987 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398991 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398994 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398996 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.398999 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399001 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399004 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399009 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399012 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399014 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399016 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 12:02:27.402264 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399019 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399021 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399024 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399027 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399029 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399032 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399034 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399037 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399042 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399044 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399047 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399050 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399053 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399055 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399058 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399061 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399063 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399066 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399069 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399071 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 12:02:27.402779 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399074 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.399080 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399332 2582 flags.go:64] FLAG: --address="0.0.0.0" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399375 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399383 2582 flags.go:64] FLAG: --anonymous-auth="true" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399388 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399392 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399396 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399400 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399405 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399408 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399411 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399414 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399418 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399421 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399423 2582 flags.go:64] FLAG: --cgroup-root="" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399426 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399429 2582 flags.go:64] FLAG: --client-ca-file="" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399432 2582 flags.go:64] FLAG: --cloud-provider="external" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399435 2582 flags.go:64] FLAG: --cluster-dns="[]" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399439 2582 flags.go:64] FLAG: --cluster-domain="" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399441 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399444 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399448 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 15 12:02:27.403288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399451 2582 flags.go:64] FLAG: --container-log-max-files="5" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399454 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399457 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399460 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399464 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399467 2582 flags.go:64] FLAG: --contention-profiling="false" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399471 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399474 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399477 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399480 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399484 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399487 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399490 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399492 2582 flags.go:64] FLAG: --enable-load-reader="false" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399495 2582 flags.go:64] FLAG: --enable-server="true" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399498 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399503 2582 flags.go:64] FLAG: --event-burst="100" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399506 2582 flags.go:64] FLAG: --event-qps="50" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399509 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399512 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399515 2582 flags.go:64] FLAG: --eviction-hard="" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399519 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399522 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399524 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399529 2582 flags.go:64] FLAG: --eviction-soft="" Sep 15 12:02:27.403872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399532 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399535 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399538 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399541 2582 flags.go:64] FLAG: --experimental-mounter-path="" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399544 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399547 2582 flags.go:64] FLAG: --fail-swap-on="true" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399550 2582 flags.go:64] FLAG: --feature-gates="" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399554 2582 flags.go:64] FLAG: --file-check-frequency="20s" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399557 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399560 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399563 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399566 2582 flags.go:64] FLAG: --healthz-port="10248" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399569 2582 flags.go:64] FLAG: --help="false" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399571 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399574 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399577 2582 flags.go:64] FLAG: --http-check-frequency="20s" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399580 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399584 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399588 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399591 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399594 2582 flags.go:64] FLAG: --image-service-endpoint="" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399597 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399600 2582 flags.go:64] FLAG: --kube-api-burst="100" Sep 15 12:02:27.404486 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399603 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399606 2582 flags.go:64] FLAG: --kube-api-qps="50" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399609 2582 flags.go:64] FLAG: --kube-reserved="" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399611 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399614 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399617 2582 flags.go:64] FLAG: --kubelet-cgroups="" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399619 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399622 2582 flags.go:64] FLAG: --lock-file="" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399630 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399633 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399636 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399642 2582 flags.go:64] FLAG: --log-json-split-stream="false" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399644 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399647 2582 flags.go:64] FLAG: --log-text-split-stream="false" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399650 2582 flags.go:64] FLAG: --logging-format="text" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399654 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399657 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399660 2582 flags.go:64] FLAG: --manifest-url="" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399667 2582 flags.go:64] FLAG: --manifest-url-header="" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399672 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399675 2582 flags.go:64] FLAG: --max-open-files="1000000" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399679 2582 flags.go:64] FLAG: --max-pods="110" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399682 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399685 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399688 2582 flags.go:64] FLAG: --memory-manager-policy="None" Sep 15 12:02:27.405087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399691 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399693 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399696 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399699 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399707 2582 flags.go:64] FLAG: --node-status-max-images="50" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399710 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399713 2582 flags.go:64] FLAG: --oom-score-adj="-999" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399716 2582 flags.go:64] FLAG: --pod-cidr="" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399718 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399724 2582 flags.go:64] FLAG: --pod-manifest-path="" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399727 2582 flags.go:64] FLAG: --pod-max-pids="-1" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399729 2582 flags.go:64] FLAG: --pods-per-core="0" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399732 2582 flags.go:64] FLAG: --port="10250" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399735 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399738 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b02b96510846d5e1" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399742 2582 flags.go:64] FLAG: --qos-reserved="" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399745 2582 flags.go:64] FLAG: --read-only-port="10255" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399748 2582 flags.go:64] FLAG: --register-node="true" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399751 2582 flags.go:64] FLAG: --register-with-taints="" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399754 2582 flags.go:64] FLAG: --registry-burst="10" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399781 2582 flags.go:64] FLAG: --registry-qps="5" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399784 2582 flags.go:64] FLAG: --reserved-cpus="" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399788 2582 flags.go:64] FLAG: --reserved-memory="" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399792 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399795 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 15 12:02:27.405701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399799 2582 flags.go:64] FLAG: --rotate-certificates="false" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399802 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399804 2582 flags.go:64] FLAG: --runonce="false" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399807 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399810 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399813 2582 flags.go:64] FLAG: --seccomp-default="false" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399816 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399819 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399822 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399825 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399828 2582 flags.go:64] FLAG: --storage-driver-password="root" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399831 2582 flags.go:64] FLAG: --storage-driver-secure="false" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399834 2582 flags.go:64] FLAG: --storage-driver-table="stats" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399836 2582 flags.go:64] FLAG: --storage-driver-user="root" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399839 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399842 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399845 2582 flags.go:64] FLAG: --system-cgroups="" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399847 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399853 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399856 2582 flags.go:64] FLAG: --tls-cert-file="" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399859 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399863 2582 flags.go:64] FLAG: --tls-min-version="" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399867 2582 flags.go:64] FLAG: --tls-private-key-file="" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399869 2582 flags.go:64] FLAG: --topology-manager-policy="none" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399872 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 15 12:02:27.406369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399875 2582 flags.go:64] FLAG: --topology-manager-scope="container" Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399877 2582 flags.go:64] FLAG: --v="2" Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399881 2582 flags.go:64] FLAG: --version="false" Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399891 2582 flags.go:64] FLAG: --vmodule="" Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399894 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.399899 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400041 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400046 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400049 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400052 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400055 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400058 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400060 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400064 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400066 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400069 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400071 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400082 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400084 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400087 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400090 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 12:02:27.407013 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400092 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400095 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400098 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400100 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400104 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400106 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400109 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400112 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400116 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400119 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400121 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400124 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400126 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400129 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400131 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400140 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400146 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400149 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400152 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400154 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 12:02:27.407563 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400157 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400160 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400162 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400164 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400167 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400170 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400172 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400174 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400177 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400179 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400182 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400184 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400187 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400189 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400191 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400194 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400196 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400199 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400201 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400205 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 12:02:27.408108 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400210 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400213 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400216 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400218 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400221 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400223 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400226 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400228 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400234 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400238 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400241 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400244 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400247 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400249 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400252 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400254 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400257 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400259 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400262 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 12:02:27.408609 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400264 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400266 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400270 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400273 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400276 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400279 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400281 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400284 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400287 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400289 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400291 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400294 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400296 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400300 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400302 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400305 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400307 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400310 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400313 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 12:02:27.409102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400315 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400318 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400322 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400325 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400328 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.400330 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.401043 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.408720 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.408735 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.409502 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.409510 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409569 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409575 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409579 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 12:02:27.409575 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409582 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409585 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409588 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409591 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409594 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409596 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409599 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409602 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409604 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409607 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409609 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409612 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409614 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409617 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409619 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409622 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409625 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409627 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409630 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409632 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 12:02:27.410061 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409635 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409642 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409644 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409647 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409649 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409652 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409654 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409657 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409659 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409662 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409665 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409667 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409670 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409673 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409676 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409679 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409681 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409685 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409689 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 12:02:27.410562 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409692 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409695 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409698 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409702 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409706 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409709 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409711 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409716 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409719 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409722 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409725 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409728 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409731 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409733 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409736 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409739 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409742 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409744 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409747 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409749 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 12:02:27.411125 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409752 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409754 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409756 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409759 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409762 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409764 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409768 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409771 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409774 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409776 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409779 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409782 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409784 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409787 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409790 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409792 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409795 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409797 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409799 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 12:02:27.411610 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409802 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409804 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409806 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409809 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409811 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409813 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409816 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409819 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409821 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409829 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409832 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409835 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409837 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409840 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409842 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409845 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409847 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 12:02:27.412101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409850 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.409855 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409993 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.409998 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410001 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410004 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410006 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410009 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410012 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410015 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410017 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410020 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410022 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410025 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 12:02:27.412520 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410027 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410030 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410032 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410035 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410037 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410040 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410042 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410044 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410047 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410049 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410052 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410055 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410058 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410060 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410063 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410065 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410068 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410071 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410073 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410076 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 12:02:27.412924 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410079 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410081 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410084 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410087 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410089 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410091 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410094 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410096 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410099 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410101 2582 feature_gate.go:328] unrecognized feature gate: Example Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410103 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410106 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410110 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410113 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410116 2582 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410119 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410121 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410124 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410127 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410129 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 12:02:27.413432 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410132 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410135 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410137 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410140 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410148 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410150 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410153 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410155 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410157 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410160 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410162 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410165 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410167 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410170 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410172 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410175 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410177 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410180 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410182 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410185 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 12:02:27.414045 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410187 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410190 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410192 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410195 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410197 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410199 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410202 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410204 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410207 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410209 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410212 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410214 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410216 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410219 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410222 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410225 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410228 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410232 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410235 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 12:02:27.414589 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410237 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410241 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410244 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410247 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410250 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410253 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410255 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:27.410258 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.410263 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.410377 2582 server.go:962] "Client rotation is on, will bootstrap in background" Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.412509 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.413426 2582 server.go:1019] "Starting client certificate rotation" Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.413535 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 15 12:02:27.415186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.413593 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 15 12:02:27.416412 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.416394 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 15 12:02:27.417584 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.417563 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 15 12:02:27.426421 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.426403 2582 log.go:25] "Validated CRI v1 runtime API" Sep 15 12:02:27.430818 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.430797 2582 log.go:25] "Validated CRI v1 image API" Sep 15 12:02:27.431685 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.431669 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 15 12:02:27.434646 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.434624 2582 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 840db9e4-0cdf-4975-8da7-12871e22f98f:/dev/nvme0n1p3 f5a10d1a-fe54-44f3-a4d1-1b1e206b395f:/dev/nvme0n1p4] Sep 15 12:02:27.434715 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.434646 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 15 12:02:27.440163 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.440045 2582 manager.go:216] Machine: {Timestamp:2026-09-15 12:02:27.438338615 +0000 UTC m=+0.284017957 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101635 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c025557a3afb152c0556b35b12696 SystemUUID:ec2c0255-57a3-afb1-52c0-556b35b12696 BootID:c988506d-43ec-486a-b418-93df09f0caf6 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:df:e3:48:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:df:e3:48:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:c5:5e:c2:4d:37 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 15 12:02:27.440163 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.440154 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 15 12:02:27.440301 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.440245 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 15 12:02:27.440301 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.440271 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 15 12:02:27.442956 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.442926 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 15 12:02:27.443097 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.442960 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-139.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 15 12:02:27.444189 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.444179 2582 topology_manager.go:138] "Creating topology manager with none policy" Sep 15 12:02:27.444223 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.444192 2582 container_manager_linux.go:309] "Creating device plugin manager" Sep 15 12:02:27.444223 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.444204 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 15 12:02:27.444670 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.444661 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 15 12:02:27.445604 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.445595 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 15 12:02:27.449246 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.449234 2582 state_mem.go:36] "Initialized new in-memory state store" Sep 15 12:02:27.449361 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.449351 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 15 12:02:27.449402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.449383 2582 kubelet.go:486] "Attempting to sync node with API server" Sep 15 12:02:27.449402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.449392 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 15 12:02:27.449469 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.449409 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 15 12:02:27.449469 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.449420 2582 kubelet.go:398] "Adding apiserver pod source" Sep 15 12:02:27.449469 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.449430 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 15 12:02:27.449822 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.449807 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 15 12:02:27.449854 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.449836 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 15 12:02:27.451872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.451854 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 15 12:02:27.454194 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.454178 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 15 12:02:27.454270 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.454211 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 15 12:02:27.456198 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456186 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 15 12:02:27.456244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456205 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 15 12:02:27.456244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456211 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 15 12:02:27.456244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456216 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 15 12:02:27.456244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456226 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 15 12:02:27.456244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456232 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 15 12:02:27.456244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456245 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 15 12:02:27.456396 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456250 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 15 12:02:27.456396 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456258 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 15 12:02:27.456396 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456264 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 15 12:02:27.456396 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456272 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 15 12:02:27.456396 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.456277 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 15 12:02:27.457607 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.457597 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 15 12:02:27.457607 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.457607 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 15 12:02:27.459501 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.459476 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-139.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 15 12:02:27.459574 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.459481 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 15 12:02:27.461581 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.461563 2582 server.go:1268] "Started kubelet" Sep 15 12:02:27.461689 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.461662 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 15 12:02:27.461755 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.461725 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 15 12:02:27.461806 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.461777 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 15 12:02:27.462685 ip-10-0-130-139 systemd[1]: Started Kubernetes Kubelet. Sep 15 12:02:27.464953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.464931 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 15 12:02:27.465264 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.465232 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hdhn9" Sep 15 12:02:27.465857 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.465829 2582 server.go:310] "Adding debug handlers to kubelet server" Sep 15 12:02:27.466989 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.466964 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 15 12:02:27.467878 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.467857 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-139.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 15 12:02:27.468801 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.467986 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-139.ec2.internal.18d57ce46c54ebce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-139.ec2.internal,UID:ip-10-0-130-139.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-139.ec2.internal,},FirstTimestamp:2026-09-15 12:02:27.461532622 +0000 UTC m=+0.307211963,LastTimestamp:2026-09-15 12:02:27.461532622 +0000 UTC m=+0.307211963,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-139.ec2.internal,}" Sep 15 12:02:27.470105 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.470089 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 15 12:02:27.470547 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.470528 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 15 12:02:27.470704 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.470670 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:27.471127 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471113 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 15 12:02:27.471127 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471129 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 15 12:02:27.471256 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471242 2582 reconstruct.go:97] "Volume reconstruction finished" Sep 15 12:02:27.471256 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471255 2582 reconciler.go:29] "Reconciler: start to sync state" Sep 15 12:02:27.471410 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471385 2582 factory.go:55] Registering systemd factory Sep 15 12:02:27.471410 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471401 2582 factory.go:223] Registration of the systemd container factory successfully Sep 15 12:02:27.471512 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471432 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hdhn9" Sep 15 12:02:27.471622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471594 2582 factory.go:153] Registering CRI-O factory Sep 15 12:02:27.471622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471615 2582 factory.go:223] Registration of the crio container factory successfully Sep 15 12:02:27.471745 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471668 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 15 12:02:27.471745 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471689 2582 factory.go:103] Registering Raw factory Sep 15 12:02:27.471745 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471704 2582 manager.go:1196] Started watching for new ooms in manager Sep 15 12:02:27.471956 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.471940 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 15 12:02:27.472685 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.472673 2582 manager.go:349] Starting recovery of all containers Sep 15 12:02:27.481119 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.481097 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 12:02:27.482739 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.482719 2582 manager.go:354] Recovery completed Sep 15 12:02:27.483859 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.483843 2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-139.ec2.internal\" not found" node="ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.484529 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.484482 2582 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 15 12:02:27.487553 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.487540 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 12:02:27.493584 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.493568 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientMemory" Sep 15 12:02:27.493651 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.493603 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 12:02:27.493651 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.493616 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientPID" Sep 15 12:02:27.494289 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.494147 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 15 12:02:27.494359 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.494296 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 15 12:02:27.494359 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.494315 2582 state_mem.go:36] "Initialized new in-memory state store" Sep 15 12:02:27.494432 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.494153 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 12:02:27.495226 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.495208 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientMemory" Sep 15 12:02:27.495321 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.495241 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 12:02:27.495321 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.495256 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientPID" Sep 15 12:02:27.495479 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.495468 2582 policy_none.go:49] "None policy: Start" Sep 15 12:02:27.495516 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.495484 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 15 12:02:27.495516 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.495495 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 15 12:02:27.496465 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.496454 2582 policy_none.go:47] "Start" Sep 15 12:02:27.529493 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.529476 2582 manager.go:337] "Starting Device Plugin manager" Sep 15 12:02:27.540435 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.529514 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 15 12:02:27.540435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.529529 2582 server.go:87] "Starting device plugin registration server" Sep 15 12:02:27.540435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.529889 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 15 12:02:27.540435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.529902 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 15 12:02:27.540435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.529995 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 15 12:02:27.540435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.530349 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 15 12:02:27.540435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.530361 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 15 12:02:27.540435 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.532552 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 15 12:02:27.540435 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.532599 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:27.614876 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.614842 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 15 12:02:27.616259 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.616237 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 15 12:02:27.616339 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.616310 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 15 12:02:27.616388 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.616346 2582 kubelet.go:2448] "Starting kubelet main sync loop" Sep 15 12:02:27.616491 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.616450 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 15 12:02:27.618448 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.618427 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 12:02:27.630454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.630431 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 12:02:27.633058 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.633041 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientMemory" Sep 15 12:02:27.633162 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.633075 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 12:02:27.633162 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.633090 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientPID" Sep 15 12:02:27.633162 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.633123 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.645161 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.645087 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.645161 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.645113 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-139.ec2.internal\": node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:27.672770 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.672745 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:27.717396 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.717361 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal"] Sep 15 12:02:27.717541 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.717481 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 12:02:27.718550 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.718534 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientMemory" Sep 15 12:02:27.718653 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.718567 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 12:02:27.718653 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.718580 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientPID" Sep 15 12:02:27.719939 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.719905 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 12:02:27.720030 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.720003 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.720089 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.720040 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 12:02:27.720748 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.720723 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientMemory" Sep 15 12:02:27.720748 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.720739 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientMemory" Sep 15 12:02:27.720890 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.720757 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 12:02:27.720890 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.720774 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientPID" Sep 15 12:02:27.720890 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.720760 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 12:02:27.720890 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.720847 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientPID" Sep 15 12:02:27.722016 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.722000 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.722071 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.722031 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 12:02:27.722786 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.722771 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientMemory" Sep 15 12:02:27.722847 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.722795 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 12:02:27.722847 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.722809 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeHasSufficientPID" Sep 15 12:02:27.745794 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.745766 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-139.ec2.internal\" not found" node="ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.750155 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.750138 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-139.ec2.internal\" not found" node="ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.773414 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.773386 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:27.872964 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.872872 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bfa66ea7f7a1bbea54e77a9cdd260313-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal\" (UID: \"bfa66ea7f7a1bbea54e77a9cdd260313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.873087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.873002 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfa66ea7f7a1bbea54e77a9cdd260313-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal\" (UID: \"bfa66ea7f7a1bbea54e77a9cdd260313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.873087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.873032 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/187d0f2e9822248b4723c1d352f18f05-config\") pod \"kube-apiserver-proxy-ip-10-0-130-139.ec2.internal\" (UID: \"187d0f2e9822248b4723c1d352f18f05\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.873930 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.873895 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:27.973658 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.973575 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bfa66ea7f7a1bbea54e77a9cdd260313-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal\" (UID: \"bfa66ea7f7a1bbea54e77a9cdd260313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.973658 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.973607 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfa66ea7f7a1bbea54e77a9cdd260313-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal\" (UID: \"bfa66ea7f7a1bbea54e77a9cdd260313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.973658 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.973624 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/187d0f2e9822248b4723c1d352f18f05-config\") pod \"kube-apiserver-proxy-ip-10-0-130-139.ec2.internal\" (UID: \"187d0f2e9822248b4723c1d352f18f05\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.973866 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.973665 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/187d0f2e9822248b4723c1d352f18f05-config\") pod \"kube-apiserver-proxy-ip-10-0-130-139.ec2.internal\" (UID: \"187d0f2e9822248b4723c1d352f18f05\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.973866 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.973695 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfa66ea7f7a1bbea54e77a9cdd260313-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal\" (UID: \"bfa66ea7f7a1bbea54e77a9cdd260313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.973866 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:27.973692 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bfa66ea7f7a1bbea54e77a9cdd260313-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal\" (UID: \"bfa66ea7f7a1bbea54e77a9cdd260313\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" Sep 15 12:02:27.974620 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:27.974606 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:28.048853 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.048811 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" Sep 15 12:02:28.054152 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.054134 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal" Sep 15 12:02:28.075366 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:28.075336 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:28.175891 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:28.175849 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:28.276410 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:28.276329 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:28.377045 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:28.377015 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:28.413574 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.413227 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 15 12:02:28.413574 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.413445 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 15 12:02:28.413574 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.413510 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 15 12:02:28.475306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.475261 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 11:57:28 +0000 UTC" deadline="2028-05-02 16:21:37.384269339 +0000 UTC" Sep 15 12:02:28.475306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.475295 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14284h19m8.908977926s" Sep 15 12:02:28.477382 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:28.477364 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:28.578007 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:28.577943 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:28.603211 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:28.603180 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa66ea7f7a1bbea54e77a9cdd260313.slice/crio-3c411f1dd2df4e1154c9234a63d533e9464c6ab4cd23892ab703b1f31e23f326 WatchSource:0}: Error finding container 3c411f1dd2df4e1154c9234a63d533e9464c6ab4cd23892ab703b1f31e23f326: Status 404 returned error can't find the container with id 3c411f1dd2df4e1154c9234a63d533e9464c6ab4cd23892ab703b1f31e23f326 Sep 15 12:02:28.603509 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:28.603487 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod187d0f2e9822248b4723c1d352f18f05.slice/crio-c4fdad6c33bd42c39a390de94c1de12fdb7176a46bd6e318f3fb7ba89b23b24e WatchSource:0}: Error finding container c4fdad6c33bd42c39a390de94c1de12fdb7176a46bd6e318f3fb7ba89b23b24e: Status 404 returned error can't find the container with id c4fdad6c33bd42c39a390de94c1de12fdb7176a46bd6e318f3fb7ba89b23b24e Sep 15 12:02:28.608633 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.608615 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 12:02:28.619891 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.619849 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" event={"ID":"bfa66ea7f7a1bbea54e77a9cdd260313","Type":"ContainerStarted","Data":"3c411f1dd2df4e1154c9234a63d533e9464c6ab4cd23892ab703b1f31e23f326"} Sep 15 12:02:28.620972 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.620924 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal" event={"ID":"187d0f2e9822248b4723c1d352f18f05","Type":"ContainerStarted","Data":"c4fdad6c33bd42c39a390de94c1de12fdb7176a46bd6e318f3fb7ba89b23b24e"} Sep 15 12:02:28.678500 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:28.678459 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:28.779014 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:28.778982 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:28.879578 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:28.879502 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-139.ec2.internal\" not found" Sep 15 12:02:28.881639 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.881620 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 12:02:28.970310 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:28.970279 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 12:02:30.059697 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.059537 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" Sep 15 12:02:30.070285 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.070257 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 15 12:02:30.071212 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.071196 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal" Sep 15 12:02:30.077514 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.077496 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 15 12:02:30.341377 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.341292 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 12:02:30.538909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.538875 2582 apiserver.go:52] "Watching apiserver" Sep 15 12:02:30.551877 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.551851 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 15 12:02:30.552287 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.552265 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-mm7pz","kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal","openshift-multus/multus-additional-cni-plugins-mlbqb","openshift-multus/network-metrics-daemon-s62bl","openshift-network-operator/iptables-alerter-zbngp","openshift-ovn-kubernetes/ovnkube-node-glzrg","kube-system/global-pull-secret-syncer-dktw7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j","openshift-cluster-node-tuning-operator/tuned-qmp6f","openshift-dns/node-resolver-5n22m","openshift-image-registry/node-ca-2pbjq","openshift-multus/multus-ksqcc","openshift-network-diagnostics/network-check-target-9mw48"] Sep 15 12:02:30.554035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.554009 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.554996 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.554976 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.558290 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.558264 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:30.558754 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.558728 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:30.559542 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.559523 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 15 12:02:30.560065 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.560045 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.561226 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.561208 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:30.561322 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.561281 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:30.561462 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.561445 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 15 12:02:30.561560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.561453 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 15 12:02:30.561724 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.561708 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 15 12:02:30.561789 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.561728 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 15 12:02:30.561789 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.561710 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 15 12:02:30.561979 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.561960 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 15 12:02:30.561979 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.561972 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cbhtj\"" Sep 15 12:02:30.563118 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.562651 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.564095 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.564080 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.565535 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.565518 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:30.566757 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.566738 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:30.566850 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.566797 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:30.568156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.568139 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.569261 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.569242 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.570402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570381 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a99cecea-5828-4a9d-a608-5a64fefb91aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.570471 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570417 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-run-systemd\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.570471 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570442 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-run-openvswitch\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.570596 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570577 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94bb3d99-1f4f-4ac2-b988-6bf69427b678-env-overrides\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.570648 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570610 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzx86\" (UniqueName: \"kubernetes.io/projected/94bb3d99-1f4f-4ac2-b988-6bf69427b678-kube-api-access-qzx86\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.570648 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570628 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:30.570741 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570647 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-system-cni-dir\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.570741 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-run-netns\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.570741 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570679 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-hostroot\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.570741 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570696 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-daemon-config\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.570741 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570717 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a99cecea-5828-4a9d-a608-5a64fefb91aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.570985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570758 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4b6w\" (UniqueName: \"kubernetes.io/projected/4a77b2b1-873f-4271-aabe-fcdff5647007-kube-api-access-s4b6w\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:30.570985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570780 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-run-ovn\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.570985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570796 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-cnibin\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.570985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570825 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a99cecea-5828-4a9d-a608-5a64fefb91aa-cnibin\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.570985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570869 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-kubelet\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.570985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570888 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-slash\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.570985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570902 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-var-lib-openvswitch\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.570985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570950 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.570991 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94bb3d99-1f4f-4ac2-b988-6bf69427b678-ovn-node-metrics-cert\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571018 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571029 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-var-lib-cni-bin\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571054 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-systemd-units\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571080 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571092 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-cni-dir\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571119 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-run-k8s-cni-cncf-io\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571145 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-var-lib-cni-multus\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571186 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-etc-kubernetes\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571205 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zglcd\" (UniqueName: \"kubernetes.io/projected/2143e3e2-6a05-4656-ac84-68b7e7189041-kube-api-access-zglcd\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571261 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a99cecea-5828-4a9d-a608-5a64fefb91aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571283 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a99cecea-5828-4a9d-a608-5a64fefb91aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571307 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-run-netns\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571324 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571326 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 15 12:02:30.571338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571347 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-etc-openvswitch\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571395 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfvqb\" (UniqueName: \"kubernetes.io/projected/a99cecea-5828-4a9d-a608-5a64fefb91aa-kube-api-access-qfvqb\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571418 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571433 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571443 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-node-log\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571457 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-log-socket\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571471 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-run-ovn-kubernetes\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571517 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-cni-bin\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571546 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94bb3d99-1f4f-4ac2-b988-6bf69427b678-ovnkube-script-lib\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571573 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2143e3e2-6a05-4656-ac84-68b7e7189041-cni-binary-copy\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571608 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-socket-dir-parent\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571645 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-conf-dir\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571672 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-cni-netd\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571704 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94bb3d99-1f4f-4ac2-b988-6bf69427b678-ovnkube-config\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571720 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-var-lib-kubelet\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571730 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571734 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-run-multus-certs\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.572009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571776 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9cdr\" (UniqueName: \"kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr\") pod \"network-check-target-9mw48\" (UID: \"e4efc2ad-2afa-4eab-9503-6b52ba8cab77\") " pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571807 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a99cecea-5828-4a9d-a608-5a64fefb91aa-system-cni-dir\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571852 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571852 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571896 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.571963 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.572155 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.572164 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.572239 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tftdz\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.572260 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.572288 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sf4bj\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.572351 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.572388 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n2b8l\"" Sep 15 12:02:30.572707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.572590 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 15 12:02:30.574175 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.574154 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wrvpq\"" Sep 15 12:02:30.574175 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.574169 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nhc29\"" Sep 15 12:02:30.574407 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.574386 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7wff7\"" Sep 15 12:02:30.574614 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.574600 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 15 12:02:30.576362 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.576343 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 15 12:02:30.577822 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.577807 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 15 12:02:30.578052 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.578035 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b6dc2\"" Sep 15 12:02:30.578260 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.578245 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 15 12:02:30.579012 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.578994 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 15 12:02:30.579095 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.579051 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 15 12:02:30.581495 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.581453 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bzc5t\"" Sep 15 12:02:30.596228 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.596201 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 15 12:02:30.661302 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.661272 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 15 12:02:30.672897 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.672866 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-sys-fs\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.672897 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.672899 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-sysctl-conf\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.673132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.672923 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-run\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.673132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.672965 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-node-log\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.672985 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-log-socket\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673001 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-run-ovn-kubernetes\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-cni-bin\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673026 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-node-log\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673081 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-log-socket\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673122 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-run-ovn-kubernetes\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673032 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-run-multus-certs\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673159 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cdr\" (UniqueName: \"kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr\") pod \"network-check-target-9mw48\" (UID: \"e4efc2ad-2afa-4eab-9503-6b52ba8cab77\") " pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673204 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-device-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673126 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-cni-bin\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673081 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-run-multus-certs\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673234 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94bb3d99-1f4f-4ac2-b988-6bf69427b678-ovnkube-config\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673270 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a99cecea-5828-4a9d-a608-5a64fefb91aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673295 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4b6w\" (UniqueName: \"kubernetes.io/projected/4a77b2b1-873f-4271-aabe-fcdff5647007-kube-api-access-s4b6w\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673325 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm856\" (UniqueName: \"kubernetes.io/projected/38db97c9-63e3-42d5-badc-c21b5cd9348a-kube-api-access-gm856\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673368 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/859d410c-5eea-4f7f-a913-f86efa9a74dc-dbus\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673391 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2843e32-3281-4cc4-a0cb-1700a9929e5a-serviceca\") pod \"node-ca-2pbjq\" (UID: \"c2843e32-3281-4cc4-a0cb-1700a9929e5a\") " pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-kubernetes\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673438 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-run-openvswitch\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673465 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a99cecea-5828-4a9d-a608-5a64fefb91aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673505 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7-konnectivity-ca\") pod \"konnectivity-agent-mm7pz\" (UID: \"40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7\") " pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673536 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-run-openvswitch\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.673568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673545 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-lib-modules\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673576 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-run-netns\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673627 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-run-netns\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673629 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-daemon-config\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b-tmp-dir\") pod \"node-resolver-5n22m\" (UID: \"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b\") " pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673692 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-run-ovn\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673717 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a99cecea-5828-4a9d-a608-5a64fefb91aa-cnibin\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673736 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-run-ovn\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673739 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-etc-selinux\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673772 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a99cecea-5828-4a9d-a608-5a64fefb91aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673782 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a99cecea-5828-4a9d-a608-5a64fefb91aa-cnibin\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673790 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-kubelet\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673814 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-kubelet\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673816 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-slash\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673841 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-var-lib-openvswitch\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673869 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673897 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94bb3d99-1f4f-4ac2-b988-6bf69427b678-ovn-node-metrics-cert\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.674366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673907 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-var-lib-openvswitch\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673919 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-etc-kubernetes\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673872 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-slash\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673936 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94bb3d99-1f4f-4ac2-b988-6bf69427b678-ovnkube-config\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673946 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673970 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-cni-dir\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673961 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673995 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-var-lib-cni-multus\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.673998 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-etc-kubernetes\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674118 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-var-lib-cni-multus\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674125 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-daemon-config\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674140 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-cni-dir\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674162 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a99cecea-5828-4a9d-a608-5a64fefb91aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674186 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfvqb\" (UniqueName: \"kubernetes.io/projected/a99cecea-5828-4a9d-a608-5a64fefb91aa-kube-api-access-qfvqb\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674192 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a99cecea-5828-4a9d-a608-5a64fefb91aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674220 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2zwd\" (UniqueName: \"kubernetes.io/projected/c2843e32-3281-4cc4-a0cb-1700a9929e5a-kube-api-access-c2zwd\") pod \"node-ca-2pbjq\" (UID: \"c2843e32-3281-4cc4-a0cb-1700a9929e5a\") " pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674246 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d234a2b8-6630-4a02-9cf9-843741ae3183-host-slash\") pod \"iptables-alerter-zbngp\" (UID: \"d234a2b8-6630-4a02-9cf9-843741ae3183\") " pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.675156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674286 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7-agent-certs\") pod \"konnectivity-agent-mm7pz\" (UID: \"40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7\") " pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674306 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-var-lib-kubelet\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674327 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-run-netns\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-etc-openvswitch\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674382 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-tmp\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674417 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-etc-openvswitch\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674418 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674432 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94bb3d99-1f4f-4ac2-b988-6bf69427b678-ovnkube-script-lib\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674418 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-run-netns\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674465 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2143e3e2-6a05-4656-ac84-68b7e7189041-cni-binary-copy\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674574 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-socket-dir-parent\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-conf-dir\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674675 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a99cecea-5828-4a9d-a608-5a64fefb91aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674682 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-socket-dir-parent\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674677 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/859d410c-5eea-4f7f-a913-f86efa9a74dc-kubelet-config\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674710 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-multus-conf-dir\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674729 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-sysctl-d\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674755 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-cni-netd\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.675874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674795 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-host-cni-netd\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674802 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-var-lib-kubelet\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674830 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a99cecea-5828-4a9d-a608-5a64fefb91aa-system-cni-dir\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674835 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-var-lib-kubelet\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-run-systemd\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674876 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a99cecea-5828-4a9d-a608-5a64fefb91aa-system-cni-dir\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674878 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94bb3d99-1f4f-4ac2-b988-6bf69427b678-env-overrides\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674911 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b-hosts-file\") pod \"node-resolver-5n22m\" (UID: \"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b\") " pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674935 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d234a2b8-6630-4a02-9cf9-843741ae3183-iptables-alerter-script\") pod \"iptables-alerter-zbngp\" (UID: \"d234a2b8-6630-4a02-9cf9-843741ae3183\") " pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674941 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94bb3d99-1f4f-4ac2-b988-6bf69427b678-ovnkube-script-lib\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674960 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzdqd\" (UniqueName: \"kubernetes.io/projected/d234a2b8-6630-4a02-9cf9-843741ae3183-kube-api-access-hzdqd\") pod \"iptables-alerter-zbngp\" (UID: \"d234a2b8-6630-4a02-9cf9-843741ae3183\") " pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674963 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-run-systemd\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.674989 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2143e3e2-6a05-4656-ac84-68b7e7189041-cni-binary-copy\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675000 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-sys\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675051 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx86\" (UniqueName: \"kubernetes.io/projected/94bb3d99-1f4f-4ac2-b988-6bf69427b678-kube-api-access-qzx86\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675071 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-system-cni-dir\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675119 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-hostroot\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.676454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675142 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-socket-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675156 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-registration-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675173 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2843e32-3281-4cc4-a0cb-1700a9929e5a-host\") pod \"node-ca-2pbjq\" (UID: \"c2843e32-3281-4cc4-a0cb-1700a9929e5a\") " pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675203 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-tuned\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675224 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-system-cni-dir\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675229 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsqmt\" (UniqueName: \"kubernetes.io/projected/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-kube-api-access-xsqmt\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675226 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94bb3d99-1f4f-4ac2-b988-6bf69427b678-env-overrides\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675265 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-cnibin\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675292 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-hostroot\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675335 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-cnibin\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675364 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-sysconfig\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675386 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-host\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675419 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-var-lib-cni-bin\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675441 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zglcd\" (UniqueName: \"kubernetes.io/projected/2143e3e2-6a05-4656-ac84-68b7e7189041-kube-api-access-zglcd\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a99cecea-5828-4a9d-a608-5a64fefb91aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675470 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64xs5\" (UniqueName: \"kubernetes.io/projected/5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b-kube-api-access-64xs5\") pod \"node-resolver-5n22m\" (UID: \"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b\") " pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-systemd-units\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.677085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675534 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-run-k8s-cni-cncf-io\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.677674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675559 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:30.677674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675582 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-modprobe-d\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.677674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675603 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-systemd\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.677674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675510 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-var-lib-cni-bin\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.677674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675680 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94bb3d99-1f4f-4ac2-b988-6bf69427b678-systemd-units\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.677674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675703 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2143e3e2-6a05-4656-ac84-68b7e7189041-host-run-k8s-cni-cncf-io\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.677674 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.675751 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:30.677674 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.675816 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs podName:4a77b2b1-873f-4271-aabe-fcdff5647007 nodeName:}" failed. No retries permitted until 2026-09-15 12:02:31.175788741 +0000 UTC m=+2.932893810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs") pod "network-metrics-daemon-s62bl" (UID: "4a77b2b1-873f-4271-aabe-fcdff5647007") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:30.677674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.675942 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a99cecea-5828-4a9d-a608-5a64fefb91aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.680078 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.680054 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94bb3d99-1f4f-4ac2-b988-6bf69427b678-ovn-node-metrics-cert\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.709618 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.709588 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 12:02:30.709618 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.709623 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 12:02:30.709618 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.709639 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v9cdr for pod openshift-network-diagnostics/network-check-target-9mw48: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:30.709877 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.709719 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr podName:e4efc2ad-2afa-4eab-9503-6b52ba8cab77 nodeName:}" failed. No retries permitted until 2026-09-15 12:02:31.20969826 +0000 UTC m=+2.966803310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v9cdr" (UniqueName: "kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr") pod "network-check-target-9mw48" (UID: "e4efc2ad-2afa-4eab-9503-6b52ba8cab77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:30.711859 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.711833 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4b6w\" (UniqueName: \"kubernetes.io/projected/4a77b2b1-873f-4271-aabe-fcdff5647007-kube-api-access-s4b6w\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:30.712004 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.711878 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzx86\" (UniqueName: \"kubernetes.io/projected/94bb3d99-1f4f-4ac2-b988-6bf69427b678-kube-api-access-qzx86\") pod \"ovnkube-node-glzrg\" (UID: \"94bb3d99-1f4f-4ac2-b988-6bf69427b678\") " pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.712306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.712285 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b7jgf" Sep 15 12:02:30.713218 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.713195 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zglcd\" (UniqueName: \"kubernetes.io/projected/2143e3e2-6a05-4656-ac84-68b7e7189041-kube-api-access-zglcd\") pod \"multus-ksqcc\" (UID: \"2143e3e2-6a05-4656-ac84-68b7e7189041\") " pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.713890 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.713859 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfvqb\" (UniqueName: \"kubernetes.io/projected/a99cecea-5828-4a9d-a608-5a64fefb91aa-kube-api-access-qfvqb\") pod \"multus-additional-cni-plugins-mlbqb\" (UID: \"a99cecea-5828-4a9d-a608-5a64fefb91aa\") " pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.723534 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.723513 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b7jgf" Sep 15 12:02:30.776205 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:30.776205 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776208 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2zwd\" (UniqueName: \"kubernetes.io/projected/c2843e32-3281-4cc4-a0cb-1700a9929e5a-kube-api-access-c2zwd\") pod \"node-ca-2pbjq\" (UID: \"c2843e32-3281-4cc4-a0cb-1700a9929e5a\") " pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776232 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d234a2b8-6630-4a02-9cf9-843741ae3183-host-slash\") pod \"iptables-alerter-zbngp\" (UID: \"d234a2b8-6630-4a02-9cf9-843741ae3183\") " pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776253 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7-agent-certs\") pod \"konnectivity-agent-mm7pz\" (UID: \"40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7\") " pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776275 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-var-lib-kubelet\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776295 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-tmp\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776318 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/859d410c-5eea-4f7f-a913-f86efa9a74dc-kubelet-config\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.776334 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776368 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-var-lib-kubelet\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:30.776401 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret podName:859d410c-5eea-4f7f-a913-f86efa9a74dc nodeName:}" failed. No retries permitted until 2026-09-15 12:02:31.27638489 +0000 UTC m=+3.033489939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret") pod "global-pull-secret-syncer-dktw7" (UID: "859d410c-5eea-4f7f-a913-f86efa9a74dc") : object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776339 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-sysctl-d\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776434 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b-hosts-file\") pod \"node-resolver-5n22m\" (UID: \"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b\") " pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.776435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776456 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d234a2b8-6630-4a02-9cf9-843741ae3183-iptables-alerter-script\") pod \"iptables-alerter-zbngp\" (UID: \"d234a2b8-6630-4a02-9cf9-843741ae3183\") " pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776471 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzdqd\" (UniqueName: \"kubernetes.io/projected/d234a2b8-6630-4a02-9cf9-843741ae3183-kube-api-access-hzdqd\") pod \"iptables-alerter-zbngp\" (UID: \"d234a2b8-6630-4a02-9cf9-843741ae3183\") " pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776513 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-sys\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776521 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-sysctl-d\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776526 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b-hosts-file\") pod \"node-resolver-5n22m\" (UID: \"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b\") " pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776538 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-socket-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776313 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d234a2b8-6630-4a02-9cf9-843741ae3183-host-slash\") pod \"iptables-alerter-zbngp\" (UID: \"d234a2b8-6630-4a02-9cf9-843741ae3183\") " pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776562 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-registration-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776617 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/859d410c-5eea-4f7f-a913-f86efa9a74dc-kubelet-config\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776618 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2843e32-3281-4cc4-a0cb-1700a9929e5a-host\") pod \"node-ca-2pbjq\" (UID: \"c2843e32-3281-4cc4-a0cb-1700a9929e5a\") " pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776654 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2843e32-3281-4cc4-a0cb-1700a9929e5a-host\") pod \"node-ca-2pbjq\" (UID: \"c2843e32-3281-4cc4-a0cb-1700a9929e5a\") " pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776706 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-registration-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776657 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-tuned\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776745 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-sys\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776747 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsqmt\" (UniqueName: \"kubernetes.io/projected/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-kube-api-access-xsqmt\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776786 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-sysconfig\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-host\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776838 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64xs5\" (UniqueName: \"kubernetes.io/projected/5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b-kube-api-access-64xs5\") pod \"node-resolver-5n22m\" (UID: \"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b\") " pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.777102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776852 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-socket-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776876 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-modprobe-d\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776899 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-systemd\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776923 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-sys-fs\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776948 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-sysctl-conf\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.776973 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-run\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777041 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-device-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm856\" (UniqueName: \"kubernetes.io/projected/38db97c9-63e3-42d5-badc-c21b5cd9348a-kube-api-access-gm856\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777092 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/859d410c-5eea-4f7f-a913-f86efa9a74dc-dbus\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777138 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2843e32-3281-4cc4-a0cb-1700a9929e5a-serviceca\") pod \"node-ca-2pbjq\" (UID: \"c2843e32-3281-4cc4-a0cb-1700a9929e5a\") " pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777164 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-kubernetes\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777194 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7-konnectivity-ca\") pod \"konnectivity-agent-mm7pz\" (UID: \"40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7\") " pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777217 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-lib-modules\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777242 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b-tmp-dir\") pod \"node-resolver-5n22m\" (UID: \"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b\") " pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777267 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-etc-selinux\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777355 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-device-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.778038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777354 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d234a2b8-6630-4a02-9cf9-843741ae3183-iptables-alerter-script\") pod \"iptables-alerter-zbngp\" (UID: \"d234a2b8-6630-4a02-9cf9-843741ae3183\") " pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777368 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777418 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-systemd\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777475 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-sysconfig\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777536 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-host\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777594 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-kubernetes\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777698 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-etc-selinux\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777723 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38db97c9-63e3-42d5-badc-c21b5cd9348a-sys-fs\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777854 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-sysctl-conf\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777875 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-lib-modules\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777905 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-run\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777926 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7-konnectivity-ca\") pod \"konnectivity-agent-mm7pz\" (UID: \"40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7\") " pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.777982 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-modprobe-d\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.778034 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/859d410c-5eea-4f7f-a913-f86efa9a74dc-dbus\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.778150 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b-tmp-dir\") pod \"node-resolver-5n22m\" (UID: \"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b\") " pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.778637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.778315 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2843e32-3281-4cc4-a0cb-1700a9929e5a-serviceca\") pod \"node-ca-2pbjq\" (UID: \"c2843e32-3281-4cc4-a0cb-1700a9929e5a\") " pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:30.779415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.779392 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-etc-tuned\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.779644 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.779623 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7-agent-certs\") pod \"konnectivity-agent-mm7pz\" (UID: \"40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7\") " pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:30.783926 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.783906 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-tmp\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.786833 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.786811 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2zwd\" (UniqueName: \"kubernetes.io/projected/c2843e32-3281-4cc4-a0cb-1700a9929e5a-kube-api-access-c2zwd\") pod \"node-ca-2pbjq\" (UID: \"c2843e32-3281-4cc4-a0cb-1700a9929e5a\") " pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:30.798102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.798071 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64xs5\" (UniqueName: \"kubernetes.io/projected/5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b-kube-api-access-64xs5\") pod \"node-resolver-5n22m\" (UID: \"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b\") " pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.798647 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.798622 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsqmt\" (UniqueName: \"kubernetes.io/projected/3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f-kube-api-access-xsqmt\") pod \"tuned-qmp6f\" (UID: \"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f\") " pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.798811 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.798791 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzdqd\" (UniqueName: \"kubernetes.io/projected/d234a2b8-6630-4a02-9cf9-843741ae3183-kube-api-access-hzdqd\") pod \"iptables-alerter-zbngp\" (UID: \"d234a2b8-6630-4a02-9cf9-843741ae3183\") " pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.799808 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.799789 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm856\" (UniqueName: \"kubernetes.io/projected/38db97c9-63e3-42d5-badc-c21b5cd9348a-kube-api-access-gm856\") pod \"aws-ebs-csi-driver-node-w7b8j\" (UID: \"38db97c9-63e3-42d5-badc-c21b5cd9348a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.830233 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.830206 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 12:02:30.866834 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.866760 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:30.875776 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.875751 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ksqcc" Sep 15 12:02:30.883528 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.883504 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" Sep 15 12:02:30.889930 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.889913 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zbngp" Sep 15 12:02:30.899109 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.899089 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" Sep 15 12:02:30.904839 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.904820 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:30.912774 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.912755 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" Sep 15 12:02:30.920065 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.920037 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5n22m" Sep 15 12:02:30.924641 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:30.924620 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pbjq" Sep 15 12:02:31.172075 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:31.171867 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2143e3e2_6a05_4656_ac84_68b7e7189041.slice/crio-33f5e67dec220c1fce0bbc8e3eeacafcb60632b31f1cc33c2b293764dd0d206f WatchSource:0}: Error finding container 33f5e67dec220c1fce0bbc8e3eeacafcb60632b31f1cc33c2b293764dd0d206f: Status 404 returned error can't find the container with id 33f5e67dec220c1fce0bbc8e3eeacafcb60632b31f1cc33c2b293764dd0d206f Sep 15 12:02:31.173901 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:31.173870 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7beb0e_ce51_4c27_97b4_33be9eb3cf5b.slice/crio-d5d6bee0c4a03648f51b98562b81e3b91e5330a8cad1145fb37447f637f92bfe WatchSource:0}: Error finding container d5d6bee0c4a03648f51b98562b81e3b91e5330a8cad1145fb37447f637f92bfe: Status 404 returned error can't find the container with id d5d6bee0c4a03648f51b98562b81e3b91e5330a8cad1145fb37447f637f92bfe Sep 15 12:02:31.175635 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:31.175584 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2843e32_3281_4cc4_a0cb_1700a9929e5a.slice/crio-87c7bd3a474b5d5ff6b5b6e8183347c7a6488ff8ab8f281b60f7280f2ac7da77 WatchSource:0}: Error finding container 87c7bd3a474b5d5ff6b5b6e8183347c7a6488ff8ab8f281b60f7280f2ac7da77: Status 404 returned error can't find the container with id 87c7bd3a474b5d5ff6b5b6e8183347c7a6488ff8ab8f281b60f7280f2ac7da77 Sep 15 12:02:31.179591 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.179525 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:31.179591 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:31.179564 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40abd97c_1d7a_4ff0_a8e1_23c8fb4805a7.slice/crio-251895b71fbe6d2049c6f8f6d36aae814e7e3b683f402f83578f91fe819658d4 WatchSource:0}: Error finding container 251895b71fbe6d2049c6f8f6d36aae814e7e3b683f402f83578f91fe819658d4: Status 404 returned error can't find the container with id 251895b71fbe6d2049c6f8f6d36aae814e7e3b683f402f83578f91fe819658d4 Sep 15 12:02:31.179756 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:31.179643 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:31.179756 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:31.179697 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs podName:4a77b2b1-873f-4271-aabe-fcdff5647007 nodeName:}" failed. No retries permitted until 2026-09-15 12:02:32.179682439 +0000 UTC m=+3.936787488 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs") pod "network-metrics-daemon-s62bl" (UID: "4a77b2b1-873f-4271-aabe-fcdff5647007") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:31.181282 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:31.181263 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94bb3d99_1f4f_4ac2_b988_6bf69427b678.slice/crio-09b4eee3d78ab8772ce3c21ef72ca5022b03c33f2f8e72eec9a6a3219cc13aa8 WatchSource:0}: Error finding container 09b4eee3d78ab8772ce3c21ef72ca5022b03c33f2f8e72eec9a6a3219cc13aa8: Status 404 returned error can't find the container with id 09b4eee3d78ab8772ce3c21ef72ca5022b03c33f2f8e72eec9a6a3219cc13aa8 Sep 15 12:02:31.184627 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:31.182920 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e450ba8_a590_4b96_a1ad_bdaa3f1ed50f.slice/crio-2444b5a864f7e91da69dbcd0f01128e13401cd0c8ceed502df8190a47511cd87 WatchSource:0}: Error finding container 2444b5a864f7e91da69dbcd0f01128e13401cd0c8ceed502df8190a47511cd87: Status 404 returned error can't find the container with id 2444b5a864f7e91da69dbcd0f01128e13401cd0c8ceed502df8190a47511cd87 Sep 15 12:02:31.184627 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:31.183929 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99cecea_5828_4a9d_a608_5a64fefb91aa.slice/crio-b9803798a319b960479979b3736d79b99112b754894f51428eb92a7987428799 WatchSource:0}: Error finding container b9803798a319b960479979b3736d79b99112b754894f51428eb92a7987428799: Status 404 returned error can't find the container with id b9803798a319b960479979b3736d79b99112b754894f51428eb92a7987428799 Sep 15 12:02:31.185852 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:02:31.185830 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38db97c9_63e3_42d5_badc_c21b5cd9348a.slice/crio-db478a66b0cf4ae1469e2a21ae4673386c258514a4a83dedf05b3c7a7a275502 WatchSource:0}: Error finding container db478a66b0cf4ae1469e2a21ae4673386c258514a4a83dedf05b3c7a7a275502: Status 404 returned error can't find the container with id db478a66b0cf4ae1469e2a21ae4673386c258514a4a83dedf05b3c7a7a275502 Sep 15 12:02:31.280463 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.280419 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:31.280639 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.280505 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cdr\" (UniqueName: \"kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr\") pod \"network-check-target-9mw48\" (UID: \"e4efc2ad-2afa-4eab-9503-6b52ba8cab77\") " pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:31.280639 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:31.280590 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:31.280639 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:31.280610 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 12:02:31.280639 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:31.280624 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 12:02:31.280639 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:31.280633 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v9cdr for pod openshift-network-diagnostics/network-check-target-9mw48: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:31.280822 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:31.280667 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret podName:859d410c-5eea-4f7f-a913-f86efa9a74dc nodeName:}" failed. No retries permitted until 2026-09-15 12:02:32.280652652 +0000 UTC m=+4.037757701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret") pod "global-pull-secret-syncer-dktw7" (UID: "859d410c-5eea-4f7f-a913-f86efa9a74dc") : object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:31.280822 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:31.280681 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr podName:e4efc2ad-2afa-4eab-9503-6b52ba8cab77 nodeName:}" failed. No retries permitted until 2026-09-15 12:02:32.280675299 +0000 UTC m=+4.037780348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9cdr" (UniqueName: "kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr") pod "network-check-target-9mw48" (UID: "e4efc2ad-2afa-4eab-9503-6b52ba8cab77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:31.715801 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.715681 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" event={"ID":"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f","Type":"ContainerStarted","Data":"2444b5a864f7e91da69dbcd0f01128e13401cd0c8ceed502df8190a47511cd87"} Sep 15 12:02:31.718438 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.718398 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" event={"ID":"94bb3d99-1f4f-4ac2-b988-6bf69427b678","Type":"ContainerStarted","Data":"09b4eee3d78ab8772ce3c21ef72ca5022b03c33f2f8e72eec9a6a3219cc13aa8"} Sep 15 12:02:31.721082 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.721052 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zbngp" event={"ID":"d234a2b8-6630-4a02-9cf9-843741ae3183","Type":"ContainerStarted","Data":"b2d7ef08fbdc0b0b9c8025324b47072a4b197dd5d04bb6078f45f66941a6dabf"} Sep 15 12:02:31.722920 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.722894 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pbjq" event={"ID":"c2843e32-3281-4cc4-a0cb-1700a9929e5a","Type":"ContainerStarted","Data":"87c7bd3a474b5d5ff6b5b6e8183347c7a6488ff8ab8f281b60f7280f2ac7da77"} Sep 15 12:02:31.724419 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.724393 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mm7pz" event={"ID":"40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7","Type":"ContainerStarted","Data":"251895b71fbe6d2049c6f8f6d36aae814e7e3b683f402f83578f91fe819658d4"} Sep 15 12:02:31.726187 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.726156 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 11:57:30 +0000 UTC" deadline="2028-04-26 05:06:59.682320196 +0000 UTC" Sep 15 12:02:31.726279 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.726194 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14129h4m27.956130686s" Sep 15 12:02:31.726986 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.726963 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5n22m" event={"ID":"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b","Type":"ContainerStarted","Data":"d5d6bee0c4a03648f51b98562b81e3b91e5330a8cad1145fb37447f637f92bfe"} Sep 15 12:02:31.729125 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.729098 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ksqcc" event={"ID":"2143e3e2-6a05-4656-ac84-68b7e7189041","Type":"ContainerStarted","Data":"33f5e67dec220c1fce0bbc8e3eeacafcb60632b31f1cc33c2b293764dd0d206f"} Sep 15 12:02:31.732424 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.732399 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal" event={"ID":"187d0f2e9822248b4723c1d352f18f05","Type":"ContainerStarted","Data":"f2f92fd2160927666beb71fcc43e0a2552443c1ce7e555671a163c3ca10d8bc4"} Sep 15 12:02:31.734776 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.734753 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" event={"ID":"38db97c9-63e3-42d5-badc-c21b5cd9348a","Type":"ContainerStarted","Data":"db478a66b0cf4ae1469e2a21ae4673386c258514a4a83dedf05b3c7a7a275502"} Sep 15 12:02:31.737674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.737653 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" event={"ID":"a99cecea-5828-4a9d-a608-5a64fefb91aa","Type":"ContainerStarted","Data":"b9803798a319b960479979b3736d79b99112b754894f51428eb92a7987428799"} Sep 15 12:02:31.747237 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:31.747192 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-139.ec2.internal" podStartSLOduration=1.747178552 podStartE2EDuration="1.747178552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:02:31.746609823 +0000 UTC m=+3.503714894" watchObservedRunningTime="2026-09-15 12:02:31.747178552 +0000 UTC m=+3.504283623" Sep 15 12:02:32.189131 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:32.189089 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:32.189334 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.189258 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:32.189399 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.189339 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs podName:4a77b2b1-873f-4271-aabe-fcdff5647007 nodeName:}" failed. No retries permitted until 2026-09-15 12:02:34.189317082 +0000 UTC m=+5.946422136 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs") pod "network-metrics-daemon-s62bl" (UID: "4a77b2b1-873f-4271-aabe-fcdff5647007") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:32.292042 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:32.291994 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cdr\" (UniqueName: \"kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr\") pod \"network-check-target-9mw48\" (UID: \"e4efc2ad-2afa-4eab-9503-6b52ba8cab77\") " pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:32.292208 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:32.292062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:32.292279 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.292207 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:32.292279 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.292273 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret podName:859d410c-5eea-4f7f-a913-f86efa9a74dc nodeName:}" failed. No retries permitted until 2026-09-15 12:02:34.292252796 +0000 UTC m=+6.049357851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret") pod "global-pull-secret-syncer-dktw7" (UID: "859d410c-5eea-4f7f-a913-f86efa9a74dc") : object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:32.292725 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.292701 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 12:02:32.292839 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.292732 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 12:02:32.292839 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.292745 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v9cdr for pod openshift-network-diagnostics/network-check-target-9mw48: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:32.292839 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.292792 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr podName:e4efc2ad-2afa-4eab-9503-6b52ba8cab77 nodeName:}" failed. No retries permitted until 2026-09-15 12:02:34.292778295 +0000 UTC m=+6.049883348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9cdr" (UniqueName: "kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr") pod "network-check-target-9mw48" (UID: "e4efc2ad-2afa-4eab-9503-6b52ba8cab77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:32.707909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:32.707641 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:32.707909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:32.707690 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:32.707909 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.707791 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:32.708415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:32.708176 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:32.708415 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.708289 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:32.708415 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:32.708392 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:32.726606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:32.726449 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 11:57:30 +0000 UTC" deadline="2028-06-04 05:42:47.539629764 +0000 UTC" Sep 15 12:02:32.726606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:32.726516 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15065h40m14.813118107s" Sep 15 12:02:34.207336 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:34.207293 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:34.207829 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.207498 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:34.207829 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.207568 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs podName:4a77b2b1-873f-4271-aabe-fcdff5647007 nodeName:}" failed. No retries permitted until 2026-09-15 12:02:38.207549288 +0000 UTC m=+9.964654353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs") pod "network-metrics-daemon-s62bl" (UID: "4a77b2b1-873f-4271-aabe-fcdff5647007") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:34.308157 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:34.308116 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cdr\" (UniqueName: \"kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr\") pod \"network-check-target-9mw48\" (UID: \"e4efc2ad-2afa-4eab-9503-6b52ba8cab77\") " pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:34.308327 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:34.308182 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:34.308327 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.308317 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:34.308449 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.308381 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret podName:859d410c-5eea-4f7f-a913-f86efa9a74dc nodeName:}" failed. No retries permitted until 2026-09-15 12:02:38.308362614 +0000 UTC m=+10.065467668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret") pod "global-pull-secret-syncer-dktw7" (UID: "859d410c-5eea-4f7f-a913-f86efa9a74dc") : object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:34.308578 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.308468 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 12:02:34.308578 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.308503 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 12:02:34.308578 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.308516 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v9cdr for pod openshift-network-diagnostics/network-check-target-9mw48: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:34.308578 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.308550 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr podName:e4efc2ad-2afa-4eab-9503-6b52ba8cab77 nodeName:}" failed. No retries permitted until 2026-09-15 12:02:38.308539195 +0000 UTC m=+10.065644251 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9cdr" (UniqueName: "kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr") pod "network-check-target-9mw48" (UID: "e4efc2ad-2afa-4eab-9503-6b52ba8cab77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:34.707916 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:34.707866 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:34.708089 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.708023 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:34.708335 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:34.708319 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:34.708430 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.708411 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:34.708507 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:34.708472 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:34.708622 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:34.708565 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:36.707630 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:36.707601 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:36.708079 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:36.707724 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:36.708079 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:36.708054 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:36.708160 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:36.708143 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:36.708210 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:36.708187 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:36.708263 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:36.708240 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:38.245297 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:38.245255 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:38.245739 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.245414 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:38.245739 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.245498 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs podName:4a77b2b1-873f-4271-aabe-fcdff5647007 nodeName:}" failed. No retries permitted until 2026-09-15 12:02:46.245462371 +0000 UTC m=+18.002567444 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs") pod "network-metrics-daemon-s62bl" (UID: "4a77b2b1-873f-4271-aabe-fcdff5647007") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:38.345776 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:38.345737 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cdr\" (UniqueName: \"kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr\") pod \"network-check-target-9mw48\" (UID: \"e4efc2ad-2afa-4eab-9503-6b52ba8cab77\") " pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:38.345958 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:38.345798 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:38.345958 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.345916 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 12:02:38.345958 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.345939 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 12:02:38.345958 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.345950 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v9cdr for pod openshift-network-diagnostics/network-check-target-9mw48: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:38.345958 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.345951 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:38.346188 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.346005 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr podName:e4efc2ad-2afa-4eab-9503-6b52ba8cab77 nodeName:}" failed. No retries permitted until 2026-09-15 12:02:46.345989859 +0000 UTC m=+18.103094909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9cdr" (UniqueName: "kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr") pod "network-check-target-9mw48" (UID: "e4efc2ad-2afa-4eab-9503-6b52ba8cab77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:38.346188 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.346021 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret podName:859d410c-5eea-4f7f-a913-f86efa9a74dc nodeName:}" failed. No retries permitted until 2026-09-15 12:02:46.346014911 +0000 UTC m=+18.103119960 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret") pod "global-pull-secret-syncer-dktw7" (UID: "859d410c-5eea-4f7f-a913-f86efa9a74dc") : object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:38.706377 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:38.706289 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:38.706560 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.706394 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:38.706730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:38.706714 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:38.706834 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.706813 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:38.706961 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:38.706943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:38.707047 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:38.707030 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:40.706185 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:40.706143 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:40.706707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:40.706146 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:40.706707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:40.706297 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:40.706707 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:40.706280 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:40.706707 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:40.706432 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:40.706707 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:40.706577 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:42.705846 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:42.705762 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:42.706287 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:42.705762 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:42.706287 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:42.705887 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:42.706287 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:42.705762 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:42.706287 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:42.705966 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:42.706287 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:42.706049 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:44.705558 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:44.705529 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:44.705993 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:44.705526 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:44.705993 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:44.705679 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:44.705993 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:44.705737 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:44.705993 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:44.705531 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:44.705993 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:44.705827 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:46.300715 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:46.300679 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:46.301231 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.300828 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:46.301231 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.300890 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs podName:4a77b2b1-873f-4271-aabe-fcdff5647007 nodeName:}" failed. No retries permitted until 2026-09-15 12:03:02.300874413 +0000 UTC m=+34.057979479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs") pod "network-metrics-daemon-s62bl" (UID: "4a77b2b1-873f-4271-aabe-fcdff5647007") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:02:46.401504 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:46.401453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cdr\" (UniqueName: \"kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr\") pod \"network-check-target-9mw48\" (UID: \"e4efc2ad-2afa-4eab-9503-6b52ba8cab77\") " pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:46.401664 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:46.401517 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:46.401664 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.401636 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:46.401664 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.401643 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 12:02:46.401799 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.401670 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 12:02:46.401799 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.401684 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v9cdr for pod openshift-network-diagnostics/network-check-target-9mw48: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:46.401799 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.401698 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret podName:859d410c-5eea-4f7f-a913-f86efa9a74dc nodeName:}" failed. No retries permitted until 2026-09-15 12:03:02.401684391 +0000 UTC m=+34.158789440 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret") pod "global-pull-secret-syncer-dktw7" (UID: "859d410c-5eea-4f7f-a913-f86efa9a74dc") : object "kube-system"/"original-pull-secret" not registered Sep 15 12:02:46.401799 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.401735 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr podName:e4efc2ad-2afa-4eab-9503-6b52ba8cab77 nodeName:}" failed. No retries permitted until 2026-09-15 12:03:02.401718339 +0000 UTC m=+34.158823403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9cdr" (UniqueName: "kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr") pod "network-check-target-9mw48" (UID: "e4efc2ad-2afa-4eab-9503-6b52ba8cab77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:02:46.705695 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:46.705617 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:46.705865 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:46.705615 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:46.705865 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.705771 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:46.705865 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:46.705631 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:46.705865 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.705834 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:46.706012 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:46.705899 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:48.706747 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:48.706706 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:48.707195 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:48.706822 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:48.707195 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:48.706833 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:48.707195 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:48.706962 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:48.707195 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:48.706966 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:48.707195 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:48.707057 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:50.706407 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.705914 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:50.707078 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.705920 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:50.707078 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.705961 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:50.707078 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:50.706535 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:50.707078 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:50.706607 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:50.707078 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:50.706478 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:50.786022 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.785957 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5n22m" event={"ID":"5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b","Type":"ContainerStarted","Data":"c519fc9c68fcfd427972fe5417ceee8c6a07e83c624ca0379a1741de5f843b91"} Sep 15 12:02:50.788066 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.788030 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ksqcc" event={"ID":"2143e3e2-6a05-4656-ac84-68b7e7189041","Type":"ContainerStarted","Data":"e30c74e143a1b26799784b3938ec591adbeab9965815b5efaad760707cfa89b1"} Sep 15 12:02:50.789791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.789764 2582 generic.go:356] "Generic (PLEG): container finished" podID="bfa66ea7f7a1bbea54e77a9cdd260313" containerID="385363c80daedbe0cfd335d17d94b7a20773574f441db14822bdfb767d7fff34" exitCode=0 Sep 15 12:02:50.789904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.789863 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" event={"ID":"bfa66ea7f7a1bbea54e77a9cdd260313","Type":"ContainerDied","Data":"385363c80daedbe0cfd335d17d94b7a20773574f441db14822bdfb767d7fff34"} Sep 15 12:02:50.791891 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.791867 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" event={"ID":"38db97c9-63e3-42d5-badc-c21b5cd9348a","Type":"ContainerStarted","Data":"5bfd260bc1b54160c5d1fa71cf9e3bcb9e64579dafd252c2e91fecb9c5e2504b"} Sep 15 12:02:50.794076 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.794053 2582 generic.go:356] "Generic (PLEG): container finished" podID="a99cecea-5828-4a9d-a608-5a64fefb91aa" containerID="f697326a68ec369885a2e0e592cb1160f41f6cf94ca942f9f7a602d5f7c51aea" exitCode=0 Sep 15 12:02:50.794190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.794138 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" event={"ID":"a99cecea-5828-4a9d-a608-5a64fefb91aa","Type":"ContainerDied","Data":"f697326a68ec369885a2e0e592cb1160f41f6cf94ca942f9f7a602d5f7c51aea"} Sep 15 12:02:50.795912 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.795880 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" event={"ID":"3e450ba8-a590-4b96-a1ad-bdaa3f1ed50f","Type":"ContainerStarted","Data":"98d10715ec06558a760027e3af1f020e65c497ede9aa68cfaf5a0e519bcf570b"} Sep 15 12:02:50.802131 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.802106 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" event={"ID":"94bb3d99-1f4f-4ac2-b988-6bf69427b678","Type":"ContainerStarted","Data":"eae019af67080cd29cb8ae28768ca68f6e35059c7492f62305f6f87e142880b3"} Sep 15 12:02:50.802231 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.802179 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" event={"ID":"94bb3d99-1f4f-4ac2-b988-6bf69427b678","Type":"ContainerStarted","Data":"ebc3c2da1b719624ca77d685878c6bb92c3daba835db61c3137b321719c20fdc"} Sep 15 12:02:50.802231 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.802220 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" event={"ID":"94bb3d99-1f4f-4ac2-b988-6bf69427b678","Type":"ContainerStarted","Data":"0eb464eae28c7055f5cdffa1fdb3d1c404eba0bd4fef66ddc5b89d318b33be8c"} Sep 15 12:02:50.802336 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.802256 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" event={"ID":"94bb3d99-1f4f-4ac2-b988-6bf69427b678","Type":"ContainerStarted","Data":"a8b1cf0df82deafda977679e4d0966d8f30fa0df6ddabc4191a11030c036bbe4"} Sep 15 12:02:50.802336 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.802301 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" event={"ID":"94bb3d99-1f4f-4ac2-b988-6bf69427b678","Type":"ContainerStarted","Data":"c0b426593703df547b96ee51cb735b08681dbf2a4f4357e3250b13816af30ed7"} Sep 15 12:02:50.802400 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.802347 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" event={"ID":"94bb3d99-1f4f-4ac2-b988-6bf69427b678","Type":"ContainerStarted","Data":"10b746401eeaa7c7daf09311bd7fb9d3e39f179baf4f8cb6af75167773f99ff0"} Sep 15 12:02:50.803818 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.803793 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pbjq" event={"ID":"c2843e32-3281-4cc4-a0cb-1700a9929e5a","Type":"ContainerStarted","Data":"33ae064e88829663cb68505cd6630ba400bd3c58812df2e92e2fca02928a29e0"} Sep 15 12:02:50.805293 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.805274 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mm7pz" event={"ID":"40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7","Type":"ContainerStarted","Data":"f3f1ae2e3901b5ac68c83f6015a02a983d1a5a4ef8d88d45b697e85ab5873256"} Sep 15 12:02:50.808070 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.808026 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5n22m" podStartSLOduration=4.401167064 podStartE2EDuration="22.808012815s" totalImagesPullingTime="18.406845751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:02:50.807290652 +0000 UTC m=+22.564395724" watchObservedRunningTime="2026-09-15 12:02:50.808012815 +0000 UTC m=+22.565117887" Sep 15 12:02:50.905551 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.905471 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qmp6f" podStartSLOduration=4.315278254 podStartE2EDuration="22.905448738s" totalImagesPullingTime="18.590170484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:02:50.883538232 +0000 UTC m=+22.640643304" watchObservedRunningTime="2026-09-15 12:02:50.905448738 +0000 UTC m=+22.662553810" Sep 15 12:02:50.906377 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.905724 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2pbjq" podStartSLOduration=4.5008673120000005 podStartE2EDuration="22.905681288s" totalImagesPullingTime="18.404813976s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:02:50.905434906 +0000 UTC m=+22.662539981" watchObservedRunningTime="2026-09-15 12:02:50.905681288 +0000 UTC m=+22.662786361" Sep 15 12:02:50.927586 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.927530 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ksqcc" podStartSLOduration=4.270106208 podStartE2EDuration="22.92750979s" totalImagesPullingTime="18.657403582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:02:50.927469257 +0000 UTC m=+22.684574331" watchObservedRunningTime="2026-09-15 12:02:50.92750979 +0000 UTC m=+22.684614856" Sep 15 12:02:50.948294 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:50.948247 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mm7pz" podStartSLOduration=12.107423209 podStartE2EDuration="22.948230785s" totalImagesPullingTime="10.840807576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:02:50.948189266 +0000 UTC m=+22.705294341" watchObservedRunningTime="2026-09-15 12:02:50.948230785 +0000 UTC m=+22.705335853" Sep 15 12:02:51.405597 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:51.405561 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 15 12:02:51.632689 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:51.632454 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T12:02:51.40557983Z","UUID":"7448e0f1-671d-4f3b-9711-1c82a15a9b41","Handler":null,"Name":"","Endpoint":""} Sep 15 12:02:51.634239 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:51.634216 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 15 12:02:51.634374 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:51.634251 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 15 12:02:51.809793 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:51.809733 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zbngp" event={"ID":"d234a2b8-6630-4a02-9cf9-843741ae3183","Type":"ContainerStarted","Data":"44bb5063a588036b15941fdce2abf15591388619a25b2e59c08c48a7e861d212"} Sep 15 12:02:51.812724 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:51.812691 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" event={"ID":"bfa66ea7f7a1bbea54e77a9cdd260313","Type":"ContainerStarted","Data":"d718311c44c76af17a96dc21fc1ae38f65a18dfbf9c50bdc71f86face81b356f"} Sep 15 12:02:51.815047 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:51.815017 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" event={"ID":"38db97c9-63e3-42d5-badc-c21b5cd9348a","Type":"ContainerStarted","Data":"d63dc01e27e0d262ea35d9cef1b20a4a1722aefac07e0b41e9624addaf6a0c98"} Sep 15 12:02:51.839430 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:51.839386 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-139.ec2.internal" podStartSLOduration=21.839371541 podStartE2EDuration="21.839371541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:02:51.838990462 +0000 UTC m=+23.596095534" watchObservedRunningTime="2026-09-15 12:02:51.839371541 +0000 UTC m=+23.596476611" Sep 15 12:02:51.839659 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:51.839630 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zbngp" podStartSLOduration=5.246163656 podStartE2EDuration="23.839618615s" totalImagesPullingTime="18.593454959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:02:51.825049937 +0000 UTC m=+23.582155008" watchObservedRunningTime="2026-09-15 12:02:51.839618615 +0000 UTC m=+23.596723683" Sep 15 12:02:52.705635 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:52.705366 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:52.705635 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:52.705464 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:52.705635 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:52.705560 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:52.705635 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:52.705606 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:52.705893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:52.705733 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:52.705893 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:52.705822 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:52.820562 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:52.819943 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" event={"ID":"38db97c9-63e3-42d5-badc-c21b5cd9348a","Type":"ContainerStarted","Data":"f04cb10b95e11fe7cf1970928218b598b3a757f6937e6a6641fc667cca07f8f6"} Sep 15 12:02:52.824013 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:52.823977 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" event={"ID":"94bb3d99-1f4f-4ac2-b988-6bf69427b678","Type":"ContainerStarted","Data":"fbe0bbed8252354d27bf2c099458a5864b3149583c73ac60c1e08402b514b8e1"} Sep 15 12:02:52.840741 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:52.840685 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w7b8j" podStartSLOduration=4.234272776 podStartE2EDuration="24.840665497s" totalImagesPullingTime="20.606392721s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:02:52.839919513 +0000 UTC m=+24.597024585" watchObservedRunningTime="2026-09-15 12:02:52.840665497 +0000 UTC m=+24.597770570" Sep 15 12:02:54.705391 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:54.705341 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:54.705832 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:54.705350 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:54.705832 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:54.705456 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:54.705832 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:54.705477 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:54.705832 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:54.705555 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:54.705832 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:54.705633 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:55.370514 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.370237 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:55.370860 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.370839 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:55.833842 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.833806 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" event={"ID":"94bb3d99-1f4f-4ac2-b988-6bf69427b678","Type":"ContainerStarted","Data":"797d13a16eb75b1f8cf162fc5a49cb1b4ee76b58edaff8378ab93a3891b77fbd"} Sep 15 12:02:55.834386 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.834058 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:55.834386 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.834126 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:55.835792 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.835768 2582 generic.go:356] "Generic (PLEG): container finished" podID="a99cecea-5828-4a9d-a608-5a64fefb91aa" containerID="e11a8c2d47f369dca6a40dbe6910e8e8e352fbdbc790fd433b414651e5048f5b" exitCode=0 Sep 15 12:02:55.835901 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.835856 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" event={"ID":"a99cecea-5828-4a9d-a608-5a64fefb91aa","Type":"ContainerDied","Data":"e11a8c2d47f369dca6a40dbe6910e8e8e352fbdbc790fd433b414651e5048f5b"} Sep 15 12:02:55.836128 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.836107 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:55.837061 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.836462 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mm7pz" Sep 15 12:02:55.851403 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.851376 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:55.864598 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:55.864552 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" podStartSLOduration=9.219194659 podStartE2EDuration="27.864537365s" totalImagesPullingTime="18.645342706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:02:55.863053994 +0000 UTC m=+27.620159057" watchObservedRunningTime="2026-09-15 12:02:55.864537365 +0000 UTC m=+27.621642436" Sep 15 12:02:56.705549 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:56.705445 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:56.705701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:56.705446 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:56.705701 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:56.705577 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:02:56.705701 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:56.705461 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:56.705701 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:56.705645 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:56.705870 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:56.705749 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:56.839817 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:56.839779 2582 generic.go:356] "Generic (PLEG): container finished" podID="a99cecea-5828-4a9d-a608-5a64fefb91aa" containerID="b7637c1390758dda2e980c4ed1c2282e7c5ac1bb1b4411c5a7067ddbd8c6df1a" exitCode=0 Sep 15 12:02:56.840377 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:56.839855 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" event={"ID":"a99cecea-5828-4a9d-a608-5a64fefb91aa","Type":"ContainerDied","Data":"b7637c1390758dda2e980c4ed1c2282e7c5ac1bb1b4411c5a7067ddbd8c6df1a"} Sep 15 12:02:56.840453 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:56.840435 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:56.856716 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:56.856694 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:02:57.844455 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:57.844423 2582 generic.go:356] "Generic (PLEG): container finished" podID="a99cecea-5828-4a9d-a608-5a64fefb91aa" containerID="16bb05861a0d1fcc5c37a3b10350cdc366467d04137c6106c8b6f7f21a50d722" exitCode=0 Sep 15 12:02:57.845094 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:57.844508 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" event={"ID":"a99cecea-5828-4a9d-a608-5a64fefb91aa","Type":"ContainerDied","Data":"16bb05861a0d1fcc5c37a3b10350cdc366467d04137c6106c8b6f7f21a50d722"} Sep 15 12:02:58.706969 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:58.706935 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:02:58.707143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:58.707006 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:02:58.707143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:02:58.707051 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:02:58.707143 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:58.707071 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:02:58.707261 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:58.707124 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:02:58.707261 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:02:58.707186 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:03:00.705743 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:00.705707 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:00.706674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:00.705719 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:00.706674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:00.705832 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:00.706674 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:00.705832 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:03:00.706674 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:00.705916 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:03:00.706674 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:00.706001 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:03:02.317979 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:02.317952 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:02.318422 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.318116 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:03:02.318422 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.318186 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs podName:4a77b2b1-873f-4271-aabe-fcdff5647007 nodeName:}" failed. No retries permitted until 2026-09-15 12:03:34.318167148 +0000 UTC m=+66.075272200 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs") pod "network-metrics-daemon-s62bl" (UID: "4a77b2b1-873f-4271-aabe-fcdff5647007") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 12:03:02.418863 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:02.418826 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:02.418966 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:02.418913 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cdr\" (UniqueName: \"kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr\") pod \"network-check-target-9mw48\" (UID: \"e4efc2ad-2afa-4eab-9503-6b52ba8cab77\") " pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:02.419023 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.418998 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 12:03:02.419071 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.419032 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 12:03:02.419071 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.419047 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 12:03:02.419071 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.419058 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v9cdr for pod openshift-network-diagnostics/network-check-target-9mw48: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:03:02.419209 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.419080 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret podName:859d410c-5eea-4f7f-a913-f86efa9a74dc nodeName:}" failed. No retries permitted until 2026-09-15 12:03:34.41905921 +0000 UTC m=+66.176164273 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret") pod "global-pull-secret-syncer-dktw7" (UID: "859d410c-5eea-4f7f-a913-f86efa9a74dc") : object "kube-system"/"original-pull-secret" not registered Sep 15 12:03:02.419209 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.419104 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr podName:e4efc2ad-2afa-4eab-9503-6b52ba8cab77 nodeName:}" failed. No retries permitted until 2026-09-15 12:03:34.419089847 +0000 UTC m=+66.176194897 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9cdr" (UniqueName: "kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr") pod "network-check-target-9mw48" (UID: "e4efc2ad-2afa-4eab-9503-6b52ba8cab77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 12:03:02.705366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:02.705279 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:02.705660 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:02.705279 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:02.705660 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.705395 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:03:02.705660 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:02.705279 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:02.705660 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.705459 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:03:02.705660 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:02.705551 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:03:02.860675 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:02.860639 2582 generic.go:356] "Generic (PLEG): container finished" podID="a99cecea-5828-4a9d-a608-5a64fefb91aa" containerID="632ecd4be891af213fe8a293ea7da0bb8d000053120b5356481d9b2b835d091a" exitCode=0 Sep 15 12:03:02.860841 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:02.860698 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" event={"ID":"a99cecea-5828-4a9d-a608-5a64fefb91aa","Type":"ContainerDied","Data":"632ecd4be891af213fe8a293ea7da0bb8d000053120b5356481d9b2b835d091a"} Sep 15 12:03:03.865505 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:03.865458 2582 generic.go:356] "Generic (PLEG): container finished" podID="a99cecea-5828-4a9d-a608-5a64fefb91aa" containerID="d35f399a4ba689adb63b5a567fe4aab78845bc9b6c7169d8edcb8c9daa4ab3cf" exitCode=0 Sep 15 12:03:03.865965 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:03.865524 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" event={"ID":"a99cecea-5828-4a9d-a608-5a64fefb91aa","Type":"ContainerDied","Data":"d35f399a4ba689adb63b5a567fe4aab78845bc9b6c7169d8edcb8c9daa4ab3cf"} Sep 15 12:03:04.710588 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:04.710311 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:04.710779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:04.710342 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:04.710779 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:04.710683 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:03:04.710779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:04.710350 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:04.710779 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:04.710749 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:03:04.710934 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:04.710816 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:03:04.871173 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:04.871143 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" event={"ID":"a99cecea-5828-4a9d-a608-5a64fefb91aa","Type":"ContainerStarted","Data":"dbac6073985bcd8d92eb24b7ea1b8b606eb00e244b5ac638387b11aef450f4a5"} Sep 15 12:03:04.905562 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:04.905518 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mlbqb" podStartSLOduration=8.341800133 podStartE2EDuration="36.905503712s" totalImagesPullingTime="28.563703579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:04.905076963 +0000 UTC m=+36.662182045" watchObservedRunningTime="2026-09-15 12:03:04.905503712 +0000 UTC m=+36.662608774" Sep 15 12:03:06.706207 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:06.706173 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:06.706643 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:06.706174 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:06.706643 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:06.706269 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:03:06.706643 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:06.706293 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:06.706643 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:06.706389 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:03:06.706643 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:06.706506 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:03:08.706273 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:08.706231 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:08.706704 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:08.706336 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:03:08.706704 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:08.706408 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:08.706704 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:08.706518 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:08.706704 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:08.706551 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:03:08.706704 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:08.706579 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:03:10.708999 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:10.708965 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:10.709403 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:10.708984 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:10.709403 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:10.708984 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:10.709403 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:10.709114 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:03:10.709403 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:10.709192 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:03:10.709403 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:10.709249 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:03:11.033298 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:11.033259 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9mw48"] Sep 15 12:03:11.033604 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:11.033588 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:11.033730 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:11.033706 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:03:11.035989 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:11.035963 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dktw7"] Sep 15 12:03:11.036117 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:11.036055 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:11.036177 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:11.036151 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:03:11.036235 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:11.036199 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s62bl"] Sep 15 12:03:11.036284 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:11.036260 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:11.036360 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:11.036339 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:03:12.708447 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:12.708415 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:12.708899 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:12.708533 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:12.708899 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:12.708538 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:03:12.708899 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:12.708665 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:03:12.708899 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:12.708679 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:12.708899 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:12.708741 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:03:14.705500 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:14.705435 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:14.705500 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:14.705467 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:14.706030 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:14.705432 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:14.706030 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:14.705573 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9mw48" podUID="e4efc2ad-2afa-4eab-9503-6b52ba8cab77" Sep 15 12:03:14.706030 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:14.705627 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dktw7" podUID="859d410c-5eea-4f7f-a913-f86efa9a74dc" Sep 15 12:03:14.706030 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:14.705717 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62bl" podUID="4a77b2b1-873f-4271-aabe-fcdff5647007" Sep 15 12:03:15.529404 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.529371 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-139.ec2.internal" event="NodeReady" Sep 15 12:03:15.529589 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.529481 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 15 12:03:15.578598 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.578511 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rz5vx"] Sep 15 12:03:15.597867 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.597836 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-j7jz2"] Sep 15 12:03:15.598006 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.597958 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rz5vx" Sep 15 12:03:15.600790 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.600769 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-swgf8\"" Sep 15 12:03:15.600922 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.600801 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 15 12:03:15.600922 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.600809 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 15 12:03:15.600922 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.600809 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 15 12:03:15.619468 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.619299 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rz5vx"] Sep 15 12:03:15.619613 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.619410 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.619613 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.619506 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j7jz2"] Sep 15 12:03:15.619729 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.619669 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x5jkb"] Sep 15 12:03:15.622084 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.622062 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 15 12:03:15.622084 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.622078 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 15 12:03:15.622247 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.622236 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 15 12:03:15.622288 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.622258 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wlcl7\"" Sep 15 12:03:15.622347 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.622325 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 15 12:03:15.646044 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.646018 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x5jkb"] Sep 15 12:03:15.646199 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.646121 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.648788 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.648766 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2gwv7\"" Sep 15 12:03:15.648788 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.648767 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 15 12:03:15.648942 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.648795 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 15 12:03:15.710836 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.710805 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51cf3230-1af6-480a-9c83-6ad54f2013cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.710836 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.710841 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhv98\" (UniqueName: \"kubernetes.io/projected/14a320c1-cabe-447c-af4c-84fc10143b41-kube-api-access-fhv98\") pod \"ingress-canary-rz5vx\" (UID: \"14a320c1-cabe-447c-af4c-84fc10143b41\") " pod="openshift-ingress-canary/ingress-canary-rz5vx" Sep 15 12:03:15.711252 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.710884 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51cf3230-1af6-480a-9c83-6ad54f2013cf-data-volume\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.711252 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.710924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctgcf\" (UniqueName: \"kubernetes.io/projected/51cf3230-1af6-480a-9c83-6ad54f2013cf-kube-api-access-ctgcf\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.711252 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.710957 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51cf3230-1af6-480a-9c83-6ad54f2013cf-crio-socket\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.711252 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.711002 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14a320c1-cabe-447c-af4c-84fc10143b41-cert\") pod \"ingress-canary-rz5vx\" (UID: \"14a320c1-cabe-447c-af4c-84fc10143b41\") " pod="openshift-ingress-canary/ingress-canary-rz5vx" Sep 15 12:03:15.711252 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.711019 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51cf3230-1af6-480a-9c83-6ad54f2013cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.812312 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812270 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhv98\" (UniqueName: \"kubernetes.io/projected/14a320c1-cabe-447c-af4c-84fc10143b41-kube-api-access-fhv98\") pod \"ingress-canary-rz5vx\" (UID: \"14a320c1-cabe-447c-af4c-84fc10143b41\") " pod="openshift-ingress-canary/ingress-canary-rz5vx" Sep 15 12:03:15.812503 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812324 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md564\" (UniqueName: \"kubernetes.io/projected/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-kube-api-access-md564\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.812545 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812479 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51cf3230-1af6-480a-9c83-6ad54f2013cf-data-volume\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.812579 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812540 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctgcf\" (UniqueName: \"kubernetes.io/projected/51cf3230-1af6-480a-9c83-6ad54f2013cf-kube-api-access-ctgcf\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.812579 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812570 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51cf3230-1af6-480a-9c83-6ad54f2013cf-crio-socket\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.812655 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812599 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-metrics-tls\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.812655 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812631 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-tmp-dir\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.812745 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14a320c1-cabe-447c-af4c-84fc10143b41-cert\") pod \"ingress-canary-rz5vx\" (UID: \"14a320c1-cabe-447c-af4c-84fc10143b41\") " pod="openshift-ingress-canary/ingress-canary-rz5vx" Sep 15 12:03:15.812784 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812762 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-config-volume\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.812833 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812788 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51cf3230-1af6-480a-9c83-6ad54f2013cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.812833 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812816 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51cf3230-1af6-480a-9c83-6ad54f2013cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.812935 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.812848 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51cf3230-1af6-480a-9c83-6ad54f2013cf-crio-socket\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.813162 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.813138 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51cf3230-1af6-480a-9c83-6ad54f2013cf-data-volume\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.813339 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.813319 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51cf3230-1af6-480a-9c83-6ad54f2013cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.816895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.816873 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51cf3230-1af6-480a-9c83-6ad54f2013cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.816980 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.816914 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14a320c1-cabe-447c-af4c-84fc10143b41-cert\") pod \"ingress-canary-rz5vx\" (UID: \"14a320c1-cabe-447c-af4c-84fc10143b41\") " pod="openshift-ingress-canary/ingress-canary-rz5vx" Sep 15 12:03:15.820443 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.820424 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhv98\" (UniqueName: \"kubernetes.io/projected/14a320c1-cabe-447c-af4c-84fc10143b41-kube-api-access-fhv98\") pod \"ingress-canary-rz5vx\" (UID: \"14a320c1-cabe-447c-af4c-84fc10143b41\") " pod="openshift-ingress-canary/ingress-canary-rz5vx" Sep 15 12:03:15.820584 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.820551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctgcf\" (UniqueName: \"kubernetes.io/projected/51cf3230-1af6-480a-9c83-6ad54f2013cf-kube-api-access-ctgcf\") pod \"insights-runtime-extractor-j7jz2\" (UID: \"51cf3230-1af6-480a-9c83-6ad54f2013cf\") " pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.908737 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.908657 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rz5vx" Sep 15 12:03:15.913928 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.913905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-config-volume\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.914050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.913964 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md564\" (UniqueName: \"kubernetes.io/projected/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-kube-api-access-md564\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.914050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.914012 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-metrics-tls\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.914050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.914034 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-tmp-dir\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.914375 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.914354 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-tmp-dir\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.914537 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.914517 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-config-volume\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.916544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.916523 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-metrics-tls\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.921551 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.921533 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md564\" (UniqueName: \"kubernetes.io/projected/1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb-kube-api-access-md564\") pod \"dns-default-x5jkb\" (UID: \"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb\") " pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:15.927407 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.927393 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j7jz2" Sep 15 12:03:15.954307 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:15.954280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:16.222252 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.222213 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rz5vx"] Sep 15 12:03:16.227327 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:16.227300 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a320c1_cabe_447c_af4c_84fc10143b41.slice/crio-2f40bbe4785d2498a684f80aeba9fc3e615866f63f1bc54612824fcb0ab219fe WatchSource:0}: Error finding container 2f40bbe4785d2498a684f80aeba9fc3e615866f63f1bc54612824fcb0ab219fe: Status 404 returned error can't find the container with id 2f40bbe4785d2498a684f80aeba9fc3e615866f63f1bc54612824fcb0ab219fe Sep 15 12:03:16.243084 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.243058 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j7jz2"] Sep 15 12:03:16.246146 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:16.246118 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51cf3230_1af6_480a_9c83_6ad54f2013cf.slice/crio-707bda56af91de0b96e9a08578bcf731e29b111dd85c3e43edc00e29a1e6d3bb WatchSource:0}: Error finding container 707bda56af91de0b96e9a08578bcf731e29b111dd85c3e43edc00e29a1e6d3bb: Status 404 returned error can't find the container with id 707bda56af91de0b96e9a08578bcf731e29b111dd85c3e43edc00e29a1e6d3bb Sep 15 12:03:16.260936 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.260911 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x5jkb"] Sep 15 12:03:16.264734 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:16.264705 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1acc9419_93ce_4bdb_8ac2_58ecc9d91bcb.slice/crio-1739398934ac0ec5c6133bbafae60c282a9ab18f1e4303e12e1aa838fe353d04 WatchSource:0}: Error finding container 1739398934ac0ec5c6133bbafae60c282a9ab18f1e4303e12e1aa838fe353d04: Status 404 returned error can't find the container with id 1739398934ac0ec5c6133bbafae60c282a9ab18f1e4303e12e1aa838fe353d04 Sep 15 12:03:16.709716 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.709640 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:16.709871 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.709640 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:16.709871 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.709644 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:16.712611 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.712477 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 15 12:03:16.713886 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.713866 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 15 12:03:16.714001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.713921 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 15 12:03:16.714001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.713925 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9l82m\"" Sep 15 12:03:16.714125 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.714105 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 15 12:03:16.714238 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.714169 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k48ps\"" Sep 15 12:03:16.903365 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.903325 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x5jkb" event={"ID":"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb","Type":"ContainerStarted","Data":"1739398934ac0ec5c6133bbafae60c282a9ab18f1e4303e12e1aa838fe353d04"} Sep 15 12:03:16.905123 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.905089 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j7jz2" event={"ID":"51cf3230-1af6-480a-9c83-6ad54f2013cf","Type":"ContainerStarted","Data":"cd7715ee43e0ded74335c21efa50a9617cf18dbe5e6a63f74bcfd03388dcdfeb"} Sep 15 12:03:16.905272 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.905141 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j7jz2" event={"ID":"51cf3230-1af6-480a-9c83-6ad54f2013cf","Type":"ContainerStarted","Data":"707bda56af91de0b96e9a08578bcf731e29b111dd85c3e43edc00e29a1e6d3bb"} Sep 15 12:03:16.906446 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:16.906386 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rz5vx" event={"ID":"14a320c1-cabe-447c-af4c-84fc10143b41","Type":"ContainerStarted","Data":"2f40bbe4785d2498a684f80aeba9fc3e615866f63f1bc54612824fcb0ab219fe"} Sep 15 12:03:17.912214 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:17.911941 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j7jz2" event={"ID":"51cf3230-1af6-480a-9c83-6ad54f2013cf","Type":"ContainerStarted","Data":"97d8683fcd92ffd2716b1342599c075c51ad30ce674f4ac7d1501bd879d0fa30"} Sep 15 12:03:18.916159 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:18.916074 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rz5vx" event={"ID":"14a320c1-cabe-447c-af4c-84fc10143b41","Type":"ContainerStarted","Data":"84b2ae1c820aaf345a0eb8240d24cc78ed23f80bbfdfefb24d226750d68589e3"} Sep 15 12:03:18.935109 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:18.935055 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rz5vx" podStartSLOduration=1.591108478 podStartE2EDuration="3.935031715s" totalImagesPullingTime="2.343923237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:18.933706342 +0000 UTC m=+50.690811414" watchObservedRunningTime="2026-09-15 12:03:18.935031715 +0000 UTC m=+50.692136789" Sep 15 12:03:19.921545 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:19.921446 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j7jz2" event={"ID":"51cf3230-1af6-480a-9c83-6ad54f2013cf","Type":"ContainerStarted","Data":"2a87d3402c00bad4a2a4e6989fbf66b852be76c4acb56c96d5c5a3328bbc0f0d"} Sep 15 12:03:19.939456 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:19.939408 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-j7jz2" podStartSLOduration=1.912604032 podStartE2EDuration="4.93939355s" totalImagesPullingTime="3.026789518s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:03:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:19.937861088 +0000 UTC m=+51.694966161" watchObservedRunningTime="2026-09-15 12:03:19.93939355 +0000 UTC m=+51.696498621" Sep 15 12:03:20.190749 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.190659 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr"] Sep 15 12:03:20.192713 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.192693 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.195275 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.195249 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 15 12:03:20.195390 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.195279 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 15 12:03:20.195390 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.195336 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 15 12:03:20.195390 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.195281 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 15 12:03:20.195390 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.195249 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 15 12:03:20.196517 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.196495 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-cl66m\"" Sep 15 12:03:20.203933 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.203914 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr"] Sep 15 12:03:20.350066 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.350034 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/96a6e508-7764-49da-8691-c28848eb19d4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.350235 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.350072 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5dqr\" (UniqueName: \"kubernetes.io/projected/96a6e508-7764-49da-8691-c28848eb19d4-kube-api-access-z5dqr\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.350235 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.350128 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96a6e508-7764-49da-8691-c28848eb19d4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.350235 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.350157 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96a6e508-7764-49da-8691-c28848eb19d4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.451405 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.451311 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96a6e508-7764-49da-8691-c28848eb19d4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.451405 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.451362 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96a6e508-7764-49da-8691-c28848eb19d4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.451645 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.451567 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/96a6e508-7764-49da-8691-c28848eb19d4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.451645 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.451603 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5dqr\" (UniqueName: \"kubernetes.io/projected/96a6e508-7764-49da-8691-c28848eb19d4-kube-api-access-z5dqr\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.451999 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.451980 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96a6e508-7764-49da-8691-c28848eb19d4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.455253 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.455160 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96a6e508-7764-49da-8691-c28848eb19d4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.455253 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.455203 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/96a6e508-7764-49da-8691-c28848eb19d4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.461141 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.461119 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5dqr\" (UniqueName: \"kubernetes.io/projected/96a6e508-7764-49da-8691-c28848eb19d4-kube-api-access-z5dqr\") pod \"prometheus-operator-68bcd7f987-hqwfr\" (UID: \"96a6e508-7764-49da-8691-c28848eb19d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.502690 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.502659 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" Sep 15 12:03:20.781000 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:20.780761 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr"] Sep 15 12:03:21.015301 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:21.015258 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a6e508_7764_49da_8691_c28848eb19d4.slice/crio-5f68fe854bc3280de791a52c0a0e299518848b3f0fa7ad1bfcc541501fc9e750 WatchSource:0}: Error finding container 5f68fe854bc3280de791a52c0a0e299518848b3f0fa7ad1bfcc541501fc9e750: Status 404 returned error can't find the container with id 5f68fe854bc3280de791a52c0a0e299518848b3f0fa7ad1bfcc541501fc9e750 Sep 15 12:03:21.930019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:21.929978 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x5jkb" event={"ID":"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb","Type":"ContainerStarted","Data":"976f96db9fb6a90aa61a563da5338e5520eb00322c0a403bb4a2677f4e008414"} Sep 15 12:03:21.930224 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:21.930045 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x5jkb" event={"ID":"1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb","Type":"ContainerStarted","Data":"4026263f7ee80f9d5855b659abbb0ee103e175ad761ed2fea565ff3ec0f79f9a"} Sep 15 12:03:21.930224 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:21.930091 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:21.931260 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:21.931228 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" event={"ID":"96a6e508-7764-49da-8691-c28848eb19d4","Type":"ContainerStarted","Data":"5f68fe854bc3280de791a52c0a0e299518848b3f0fa7ad1bfcc541501fc9e750"} Sep 15 12:03:21.948014 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:21.947969 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x5jkb" podStartSLOduration=2.176458731 podStartE2EDuration="6.947952357s" totalImagesPullingTime="4.771493626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:21.946431129 +0000 UTC m=+53.703536201" watchObservedRunningTime="2026-09-15 12:03:21.947952357 +0000 UTC m=+53.705057429" Sep 15 12:03:22.936403 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:22.936357 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" event={"ID":"96a6e508-7764-49da-8691-c28848eb19d4","Type":"ContainerStarted","Data":"61d8ca667c9e3d767fff69380c672a14368b56d36a98edaff2e6ef617a0f4402"} Sep 15 12:03:22.937074 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:22.936425 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" event={"ID":"96a6e508-7764-49da-8691-c28848eb19d4","Type":"ContainerStarted","Data":"539c98235ddc1a2fa15e15a0141aa30a33ab07182fd9042b62a3247b6c2d31bc"} Sep 15 12:03:22.953789 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:22.953733 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hqwfr" podStartSLOduration=1.70811053 podStartE2EDuration="2.953714316s" totalImagesPullingTime="1.245603786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:22.953262507 +0000 UTC m=+54.710367579" watchObservedRunningTime="2026-09-15 12:03:22.953714316 +0000 UTC m=+54.710819390" Sep 15 12:03:23.376870 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.376832 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fd7f7dd8f-ntcpv"] Sep 15 12:03:23.378548 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.378533 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.381415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.381397 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 15 12:03:23.382526 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.382511 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 15 12:03:23.382614 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.382530 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 15 12:03:23.382614 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.382561 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 15 12:03:23.382775 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.382761 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 15 12:03:23.382821 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.382782 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qvfg6\"" Sep 15 12:03:23.382821 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.382789 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 15 12:03:23.382900 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.382882 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 15 12:03:23.417501 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.417459 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fd7f7dd8f-ntcpv"] Sep 15 12:03:23.577650 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.577614 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-oauth-serving-cert\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.577650 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.577652 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-service-ca\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.577860 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.577672 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-console-config\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.577860 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.577751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-serving-cert\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.577860 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.577792 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nhgd\" (UniqueName: \"kubernetes.io/projected/cffc8507-c99a-4683-91e0-0daf66174291-kube-api-access-9nhgd\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.577860 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.577844 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-oauth-config\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.679011 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.678910 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-oauth-config\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.679011 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.678986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-oauth-serving-cert\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.679011 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.679013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-service-ca\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.679257 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.679031 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-console-config\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.679257 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.679072 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-serving-cert\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.679257 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.679095 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhgd\" (UniqueName: \"kubernetes.io/projected/cffc8507-c99a-4683-91e0-0daf66174291-kube-api-access-9nhgd\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.679784 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.679760 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-console-config\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.679893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.679820 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-service-ca\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.679893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.679842 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-oauth-serving-cert\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.682244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.682221 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-oauth-config\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.682341 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.682303 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-serving-cert\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.687390 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.687371 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nhgd\" (UniqueName: \"kubernetes.io/projected/cffc8507-c99a-4683-91e0-0daf66174291-kube-api-access-9nhgd\") pod \"console-fd7f7dd8f-ntcpv\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.687526 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.687513 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:23.959327 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:23.959248 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fd7f7dd8f-ntcpv"] Sep 15 12:03:23.962123 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:23.962098 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcffc8507_c99a_4683_91e0_0daf66174291.slice/crio-d688258be5ce11666e4b76db74d682f4f67266f32012ba7619e4a042aa1ef528 WatchSource:0}: Error finding container d688258be5ce11666e4b76db74d682f4f67266f32012ba7619e4a042aa1ef528: Status 404 returned error can't find the container with id d688258be5ce11666e4b76db74d682f4f67266f32012ba7619e4a042aa1ef528 Sep 15 12:03:24.552592 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.552556 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs"] Sep 15 12:03:24.554838 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.554822 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.557454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.557423 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 15 12:03:24.557454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.557440 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 15 12:03:24.557713 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.557698 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-spfx5\"" Sep 15 12:03:24.571670 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.571646 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs"] Sep 15 12:03:24.572549 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.572526 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lfvs8"] Sep 15 12:03:24.574800 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.574779 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.577581 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.577562 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bh5pf\"" Sep 15 12:03:24.577768 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.577755 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 15 12:03:24.577882 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.577855 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 15 12:03:24.577993 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.577931 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 15 12:03:24.585300 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585281 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.585391 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585315 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-wtmp\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.585391 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585338 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ef1b202-71f6-4738-9054-57aa0dea17fd-root\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.585391 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585369 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.585512 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585400 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-textfile\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.585512 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585414 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ef1b202-71f6-4738-9054-57aa0dea17fd-metrics-client-ca\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.585512 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585441 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-565cp\" (UniqueName: \"kubernetes.io/projected/7ef1b202-71f6-4738-9054-57aa0dea17fd-kube-api-access-565cp\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.585512 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585465 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w7b4\" (UniqueName: \"kubernetes.io/projected/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-kube-api-access-7w7b4\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.585663 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585541 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.585663 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585585 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-tls\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.585663 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585601 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ef1b202-71f6-4738-9054-57aa0dea17fd-sys\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.585663 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585625 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.585663 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.585649 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.686806 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.686771 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ef1b202-71f6-4738-9054-57aa0dea17fd-sys\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.686966 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.686844 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.686966 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.686872 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.686966 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.686888 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ef1b202-71f6-4738-9054-57aa0dea17fd-sys\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.686966 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.686895 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.687174 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.686985 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-wtmp\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.687174 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ef1b202-71f6-4738-9054-57aa0dea17fd-root\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.687174 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687045 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.687174 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687087 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-textfile\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.687367 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687216 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-wtmp\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.687367 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687246 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ef1b202-71f6-4738-9054-57aa0dea17fd-metrics-client-ca\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.687367 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:24.687287 2582 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 15 12:03:24.687367 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-565cp\" (UniqueName: \"kubernetes.io/projected/7ef1b202-71f6-4738-9054-57aa0dea17fd-kube-api-access-565cp\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.687367 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687329 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w7b4\" (UniqueName: \"kubernetes.io/projected/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-kube-api-access-7w7b4\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.687625 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:03:24.687379 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-openshift-state-metrics-tls podName:ef3d2d84-b9a3-4983-b3da-a7732cf7c12a nodeName:}" failed. No retries permitted until 2026-09-15 12:03:25.187358592 +0000 UTC m=+56.944463654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-47rvs" (UID: "ef3d2d84-b9a3-4983-b3da-a7732cf7c12a") : secret "openshift-state-metrics-tls" not found Sep 15 12:03:24.687625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687438 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.687625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687479 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-tls\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.687625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687526 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-textfile\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.687830 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687666 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.687940 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.687918 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ef1b202-71f6-4738-9054-57aa0dea17fd-metrics-client-ca\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.688553 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.688519 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.688670 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.688624 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ef1b202-71f6-4738-9054-57aa0dea17fd-root\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.690038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.689992 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.690378 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.690355 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.691064 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.691043 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ef1b202-71f6-4738-9054-57aa0dea17fd-node-exporter-tls\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.696525 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.696475 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-565cp\" (UniqueName: \"kubernetes.io/projected/7ef1b202-71f6-4738-9054-57aa0dea17fd-kube-api-access-565cp\") pod \"node-exporter-lfvs8\" (UID: \"7ef1b202-71f6-4738-9054-57aa0dea17fd\") " pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.697142 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.697112 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w7b4\" (UniqueName: \"kubernetes.io/projected/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-kube-api-access-7w7b4\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:24.884159 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.884087 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lfvs8" Sep 15 12:03:24.898531 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:24.897853 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef1b202_71f6_4738_9054_57aa0dea17fd.slice/crio-06a873fbdab69d94fb28e1daa6f7813896c9ff815cb002bc43334dda1012ea80 WatchSource:0}: Error finding container 06a873fbdab69d94fb28e1daa6f7813896c9ff815cb002bc43334dda1012ea80: Status 404 returned error can't find the container with id 06a873fbdab69d94fb28e1daa6f7813896c9ff815cb002bc43334dda1012ea80 Sep 15 12:03:24.945035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.944990 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lfvs8" event={"ID":"7ef1b202-71f6-4738-9054-57aa0dea17fd","Type":"ContainerStarted","Data":"06a873fbdab69d94fb28e1daa6f7813896c9ff815cb002bc43334dda1012ea80"} Sep 15 12:03:24.946391 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:24.946360 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd7f7dd8f-ntcpv" event={"ID":"cffc8507-c99a-4683-91e0-0daf66174291","Type":"ContainerStarted","Data":"d688258be5ce11666e4b76db74d682f4f67266f32012ba7619e4a042aa1ef528"} Sep 15 12:03:25.191897 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.191804 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:25.194632 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.194607 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef3d2d84-b9a3-4983-b3da-a7732cf7c12a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-47rvs\" (UID: \"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:25.464371 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.464329 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" Sep 15 12:03:25.622761 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.622714 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 12:03:25.658603 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.658572 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 12:03:25.658776 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.658712 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.662035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.661412 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 15 12:03:25.662035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.661412 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 15 12:03:25.662035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.661702 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 15 12:03:25.662035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.661749 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 15 12:03:25.662035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.661803 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 15 12:03:25.662035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.661707 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 15 12:03:25.662035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.661924 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-62dhh\"" Sep 15 12:03:25.662035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.661941 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 15 12:03:25.662417 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.662158 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 15 12:03:25.662417 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.662324 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 15 12:03:25.696685 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.696652 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxwgx\" (UniqueName: \"kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-kube-api-access-qxwgx\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.696830 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.696699 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.696830 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.696790 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-out\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.696830 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.696817 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.696948 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.696846 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.696948 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.696877 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-web-config\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.696948 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.696914 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.696948 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.696935 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.697099 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.696977 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.697099 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.697013 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.697099 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.697055 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-volume\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.697099 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.697074 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.697217 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.697140 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.770286 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.769042 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs"] Sep 15 12:03:25.797891 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.797821 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.797891 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.797867 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-volume\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.797891 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.797896 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.798182 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.797948 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.798182 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.797974 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxwgx\" (UniqueName: \"kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-kube-api-access-qxwgx\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.798182 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.797993 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.798182 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.798038 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-out\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.798182 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.798066 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.798182 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.798090 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.798182 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.798107 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-web-config\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.798182 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.798140 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.798182 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.798159 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.798679 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.798188 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.802338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.802287 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.818572 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.803164 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.818572 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.803572 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.818572 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.805420 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.818572 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.805539 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.818572 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.805973 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.818572 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.806146 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.818572 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.807956 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-web-config\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.818572 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.809389 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.818572 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.811196 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxwgx\" (UniqueName: \"kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-kube-api-access-qxwgx\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.819018 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.818715 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-out\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.819439 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.819400 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.819983 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.819960 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-volume\") pod \"alertmanager-main-0\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.970237 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:25.970207 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:03:25.970529 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:25.970503 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef3d2d84_b9a3_4983_b3da_a7732cf7c12a.slice/crio-15eff2bbd739ff1882f75f83bb712c8e9d6ebd289e9e7b7b0f3d68077e081b5a WatchSource:0}: Error finding container 15eff2bbd739ff1882f75f83bb712c8e9d6ebd289e9e7b7b0f3d68077e081b5a: Status 404 returned error can't find the container with id 15eff2bbd739ff1882f75f83bb712c8e9d6ebd289e9e7b7b0f3d68077e081b5a Sep 15 12:03:26.959025 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:26.958982 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" event={"ID":"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a","Type":"ContainerStarted","Data":"15eff2bbd739ff1882f75f83bb712c8e9d6ebd289e9e7b7b0f3d68077e081b5a"} Sep 15 12:03:27.341385 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:27.341359 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 12:03:27.342587 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:27.342562 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8334ab_f85d_46e0_8e09_57d1d497421b.slice/crio-c590cd41c73017c3fad06b79558a50f2bebc08dde77954dfdde62cc33fbca6db WatchSource:0}: Error finding container c590cd41c73017c3fad06b79558a50f2bebc08dde77954dfdde62cc33fbca6db: Status 404 returned error can't find the container with id c590cd41c73017c3fad06b79558a50f2bebc08dde77954dfdde62cc33fbca6db Sep 15 12:03:27.968330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:27.968289 2582 generic.go:356] "Generic (PLEG): container finished" podID="7ef1b202-71f6-4738-9054-57aa0dea17fd" containerID="fa24a123846af69abd739223e9a76cdb3199184cbdad6db90fb3f520d8e42698" exitCode=0 Sep 15 12:03:27.968330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:27.968314 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lfvs8" event={"ID":"7ef1b202-71f6-4738-9054-57aa0dea17fd","Type":"ContainerDied","Data":"fa24a123846af69abd739223e9a76cdb3199184cbdad6db90fb3f520d8e42698"} Sep 15 12:03:27.971021 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:27.970990 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd7f7dd8f-ntcpv" event={"ID":"cffc8507-c99a-4683-91e0-0daf66174291","Type":"ContainerStarted","Data":"c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4"} Sep 15 12:03:27.972461 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:27.972431 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerStarted","Data":"c590cd41c73017c3fad06b79558a50f2bebc08dde77954dfdde62cc33fbca6db"} Sep 15 12:03:27.974989 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:27.974961 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" event={"ID":"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a","Type":"ContainerStarted","Data":"f8c4dd5cf23629a2136a6641ece82ce61b685e500a5e9bf759781f3748281953"} Sep 15 12:03:27.975104 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:27.975017 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" event={"ID":"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a","Type":"ContainerStarted","Data":"3a46af0e1a6cbfa5f6238fbafae177bc8874683bb18aff14c6f623679a45fcb3"} Sep 15 12:03:28.005422 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.004996 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fd7f7dd8f-ntcpv" podStartSLOduration=1.8989202280000002 podStartE2EDuration="5.004927291s" totalImagesPullingTime="3.106007063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:28.003622414 +0000 UTC m=+59.760727558" watchObservedRunningTime="2026-09-15 12:03:28.004927291 +0000 UTC m=+59.762032359" Sep 15 12:03:28.858548 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.858510 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-glzrg" Sep 15 12:03:28.961731 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.961691 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6ccf494cc7-zjzqk"] Sep 15 12:03:28.964108 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.964084 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:28.966617 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.966581 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-781s14v4upsra\"" Sep 15 12:03:28.966758 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.966697 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 15 12:03:28.966758 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.966727 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-tl4n6\"" Sep 15 12:03:28.966880 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.966813 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 15 12:03:28.967179 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.967160 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 15 12:03:28.967252 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.967200 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 15 12:03:28.974622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.974603 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6ccf494cc7-zjzqk"] Sep 15 12:03:28.981737 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.981710 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lfvs8" event={"ID":"7ef1b202-71f6-4738-9054-57aa0dea17fd","Type":"ContainerStarted","Data":"ee6a95838c3e3b909cdd424c455aaf01a60644f137ef0bdb718fbccf4c27f1e8"} Sep 15 12:03:28.981863 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.981769 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lfvs8" event={"ID":"7ef1b202-71f6-4738-9054-57aa0dea17fd","Type":"ContainerStarted","Data":"d19f2c26dc953282b02a4cf070cd54db02f781e2239bd8fe3e6a91b70d259475"} Sep 15 12:03:28.983340 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.983314 2582 generic.go:356] "Generic (PLEG): container finished" podID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerID="f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1" exitCode=0 Sep 15 12:03:28.983440 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.983353 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerDied","Data":"f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1"} Sep 15 12:03:28.985801 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:28.985775 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" event={"ID":"ef3d2d84-b9a3-4983-b3da-a7732cf7c12a","Type":"ContainerStarted","Data":"6e3467db6e529ae2f3406951e940ac012d2c662a36d798b107c83148dd49bb12"} Sep 15 12:03:29.027725 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.027685 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/141bbde0-f532-43df-b468-f006caafcc52-secret-metrics-server-client-certs\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.027897 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.027843 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/141bbde0-f532-43df-b468-f006caafcc52-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.027982 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.027935 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/141bbde0-f532-43df-b468-f006caafcc52-secret-metrics-server-tls\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.028037 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.027989 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/141bbde0-f532-43df-b468-f006caafcc52-metrics-server-audit-profiles\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.028126 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.028096 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/141bbde0-f532-43df-b468-f006caafcc52-audit-log\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.028194 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.028171 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/141bbde0-f532-43df-b468-f006caafcc52-client-ca-bundle\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.028654 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.028624 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdbhd\" (UniqueName: \"kubernetes.io/projected/141bbde0-f532-43df-b468-f006caafcc52-kube-api-access-hdbhd\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.038351 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.038307 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47rvs" podStartSLOduration=3.785105133 podStartE2EDuration="5.038292832s" totalImagesPullingTime="1.253187699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:29.036930092 +0000 UTC m=+60.794035163" watchObservedRunningTime="2026-09-15 12:03:29.038292832 +0000 UTC m=+60.795397903" Sep 15 12:03:29.057106 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.057056 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lfvs8" podStartSLOduration=2.815100698 podStartE2EDuration="5.057040951s" totalImagesPullingTime="2.160795346s" totalInitContainerRuntime="81.144907ms" isStatefulPod=true podCreationTimestamp="2026-09-15 12:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:29.056265627 +0000 UTC m=+60.813370695" watchObservedRunningTime="2026-09-15 12:03:29.057040951 +0000 UTC m=+60.814146023" Sep 15 12:03:29.130090 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.129993 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdbhd\" (UniqueName: \"kubernetes.io/projected/141bbde0-f532-43df-b468-f006caafcc52-kube-api-access-hdbhd\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.130090 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.130063 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/141bbde0-f532-43df-b468-f006caafcc52-secret-metrics-server-client-certs\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.130090 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.130090 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/141bbde0-f532-43df-b468-f006caafcc52-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.130402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.130129 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/141bbde0-f532-43df-b468-f006caafcc52-secret-metrics-server-tls\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.130402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.130152 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/141bbde0-f532-43df-b468-f006caafcc52-metrics-server-audit-profiles\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.130402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.130183 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/141bbde0-f532-43df-b468-f006caafcc52-audit-log\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.130402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.130212 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/141bbde0-f532-43df-b468-f006caafcc52-client-ca-bundle\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.130755 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.130726 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/141bbde0-f532-43df-b468-f006caafcc52-audit-log\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.131138 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.131111 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/141bbde0-f532-43df-b468-f006caafcc52-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.131248 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.131231 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/141bbde0-f532-43df-b468-f006caafcc52-metrics-server-audit-profiles\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.132731 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.132703 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/141bbde0-f532-43df-b468-f006caafcc52-secret-metrics-server-client-certs\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.132817 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.132771 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/141bbde0-f532-43df-b468-f006caafcc52-client-ca-bundle\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.132817 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.132809 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/141bbde0-f532-43df-b468-f006caafcc52-secret-metrics-server-tls\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.139769 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.139752 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdbhd\" (UniqueName: \"kubernetes.io/projected/141bbde0-f532-43df-b468-f006caafcc52-kube-api-access-hdbhd\") pod \"metrics-server-6ccf494cc7-zjzqk\" (UID: \"141bbde0-f532-43df-b468-f006caafcc52\") " pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.273392 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.273352 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:29.547642 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.547233 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6ccf494cc7-zjzqk"] Sep 15 12:03:29.549842 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:29.549809 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141bbde0_f532_43df_b468_f006caafcc52.slice/crio-7130ea3933b4a169e605f1b99681087771f6f90599c8dda80c24d0586745117f WatchSource:0}: Error finding container 7130ea3933b4a169e605f1b99681087771f6f90599c8dda80c24d0586745117f: Status 404 returned error can't find the container with id 7130ea3933b4a169e605f1b99681087771f6f90599c8dda80c24d0586745117f Sep 15 12:03:29.991983 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:29.991852 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" event={"ID":"141bbde0-f532-43df-b468-f006caafcc52","Type":"ContainerStarted","Data":"7130ea3933b4a169e605f1b99681087771f6f90599c8dda80c24d0586745117f"} Sep 15 12:03:30.769079 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.768999 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 12:03:30.775521 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.773420 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.776269 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.776124 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1in3jebbq3v49\"" Sep 15 12:03:30.776791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.776767 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 15 12:03:30.776791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.776781 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 15 12:03:30.776955 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.776792 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 15 12:03:30.777011 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.776993 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 15 12:03:30.777161 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.777145 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 15 12:03:30.777222 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.777167 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 15 12:03:30.777305 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.777289 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 15 12:03:30.777364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.777317 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 15 12:03:30.777742 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.777723 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 15 12:03:30.777742 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.777740 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 15 12:03:30.778001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.777984 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 15 12:03:30.778237 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.778208 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-lrpq6\"" Sep 15 12:03:30.779068 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.779050 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 15 12:03:30.780944 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.780922 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 15 12:03:30.785309 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.784909 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 15 12:03:30.791499 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.791458 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 12:03:30.848396 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848363 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848620 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848404 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848620 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848449 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848620 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848473 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848620 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848547 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848620 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848582 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848684 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848700 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848714 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848730 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848792 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848820 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848845 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848871 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp6qs\" (UniqueName: \"kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-kube-api-access-tp6qs\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.848909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848898 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.849443 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848926 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.849443 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848948 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.849443 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.848961 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949472 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949430 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949480 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949533 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949584 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949608 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949630 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949650 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949674 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949698 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949724 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949746 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949767 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949793 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp6qs\" (UniqueName: \"kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-kube-api-access-tp6qs\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949817 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949846 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949874 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.949904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949888 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.950249 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.950249 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.949978 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.951594 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.950578 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.951594 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.950665 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.951594 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.951467 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.951829 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.951718 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.952773 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.952178 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.952773 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.952415 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.952773 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.952549 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.952997 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.952794 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.953731 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.953702 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.954036 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.953989 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.954374 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.954331 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.955697 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.955655 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.955955 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.955915 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.956042 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.955972 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.956637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.956575 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.956805 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.956782 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.957014 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.956975 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.959328 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.958879 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.959328 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.958961 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:30.961127 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:30.961106 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp6qs\" (UniqueName: \"kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-kube-api-access-tp6qs\") pod \"prometheus-k8s-0\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:31.006023 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:31.005990 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerStarted","Data":"6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40"} Sep 15 12:03:31.006470 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:31.006086 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerStarted","Data":"23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b"} Sep 15 12:03:31.086560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:31.086454 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:31.415040 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:31.415008 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 12:03:31.417917 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:31.417894 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207c7d00_7e67_44f4_bfe8_2e05b94378a6.slice/crio-4b427e3bc7ee8d5afd1c84b9dde1d1cd6b6644bedbc857af883c7485a55d8047 WatchSource:0}: Error finding container 4b427e3bc7ee8d5afd1c84b9dde1d1cd6b6644bedbc857af883c7485a55d8047: Status 404 returned error can't find the container with id 4b427e3bc7ee8d5afd1c84b9dde1d1cd6b6644bedbc857af883c7485a55d8047 Sep 15 12:03:31.939098 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:31.939069 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x5jkb" Sep 15 12:03:32.011368 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.011335 2582 generic.go:356] "Generic (PLEG): container finished" podID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerID="fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555" exitCode=0 Sep 15 12:03:32.011857 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.011431 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerDied","Data":"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555"} Sep 15 12:03:32.011857 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.011521 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerStarted","Data":"4b427e3bc7ee8d5afd1c84b9dde1d1cd6b6644bedbc857af883c7485a55d8047"} Sep 15 12:03:32.016155 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.016108 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerStarted","Data":"5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da"} Sep 15 12:03:32.016269 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.016195 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerStarted","Data":"c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d"} Sep 15 12:03:32.016269 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.016241 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerStarted","Data":"a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c"} Sep 15 12:03:32.018352 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.017925 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" event={"ID":"141bbde0-f532-43df-b468-f006caafcc52","Type":"ContainerStarted","Data":"41baff8e213ac0b3d5463a5abaa4b0fe3c80b7516f75f01a667c5515f2137c3b"} Sep 15 12:03:32.090398 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.090353 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" podStartSLOduration=2.459618435 podStartE2EDuration="4.090335452s" totalImagesPullingTime="1.630717017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:32.089521345 +0000 UTC m=+63.846626417" watchObservedRunningTime="2026-09-15 12:03:32.090335452 +0000 UTC m=+63.847440522" Sep 15 12:03:32.760205 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.760168 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c559c8986-cl7kk"] Sep 15 12:03:32.789866 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.789828 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c559c8986-cl7kk"] Sep 15 12:03:32.790049 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.789966 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.798236 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.798214 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 15 12:03:32.871616 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.871565 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-serving-cert\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.871788 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.871721 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-oauth-config\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.871788 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.871760 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-oauth-serving-cert\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.871886 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.871793 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w2z4\" (UniqueName: \"kubernetes.io/projected/390b4401-d041-446c-8277-5da3a61a8ef3-kube-api-access-2w2z4\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.871932 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.871846 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-service-ca\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.871978 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.871951 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-console-config\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.871978 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.871975 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-trusted-ca-bundle\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.973128 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.973087 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-oauth-serving-cert\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.973128 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.973136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w2z4\" (UniqueName: \"kubernetes.io/projected/390b4401-d041-446c-8277-5da3a61a8ef3-kube-api-access-2w2z4\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.973379 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.973168 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-service-ca\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.973379 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.973229 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-console-config\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.973379 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.973247 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-trusted-ca-bundle\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.973379 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.973357 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-serving-cert\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.973720 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.973426 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-oauth-config\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.973883 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.973860 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-oauth-serving-cert\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.974262 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.974240 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-console-config\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.974385 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.974338 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-service-ca\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.974576 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.974551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-trusted-ca-bundle\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.976260 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.976238 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-oauth-config\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.976357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.976335 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-serving-cert\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:32.981474 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:32.981453 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w2z4\" (UniqueName: \"kubernetes.io/projected/390b4401-d041-446c-8277-5da3a61a8ef3-kube-api-access-2w2z4\") pod \"console-c559c8986-cl7kk\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:33.100428 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:33.100339 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:33.406458 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:33.406431 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c559c8986-cl7kk"] Sep 15 12:03:33.408790 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:33.408759 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390b4401_d041_446c_8277_5da3a61a8ef3.slice/crio-dea7604b031cdaafac415a7d509b0483740a0f3e85eb178d62522e77b4fc68ea WatchSource:0}: Error finding container dea7604b031cdaafac415a7d509b0483740a0f3e85eb178d62522e77b4fc68ea: Status 404 returned error can't find the container with id dea7604b031cdaafac415a7d509b0483740a0f3e85eb178d62522e77b4fc68ea Sep 15 12:03:33.688104 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:33.688011 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:33.688104 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:33.688079 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:33.693723 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:33.693693 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:34.028584 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.028538 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c559c8986-cl7kk" event={"ID":"390b4401-d041-446c-8277-5da3a61a8ef3","Type":"ContainerStarted","Data":"dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0"} Sep 15 12:03:34.028788 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.028615 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c559c8986-cl7kk" event={"ID":"390b4401-d041-446c-8277-5da3a61a8ef3","Type":"ContainerStarted","Data":"dea7604b031cdaafac415a7d509b0483740a0f3e85eb178d62522e77b4fc68ea"} Sep 15 12:03:34.032981 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.032953 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:03:34.046650 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.046594 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c559c8986-cl7kk" podStartSLOduration=2.046573213 podStartE2EDuration="2.046573213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:03:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:34.045363698 +0000 UTC m=+65.802468780" watchObservedRunningTime="2026-09-15 12:03:34.046573213 +0000 UTC m=+65.803678287" Sep 15 12:03:34.389120 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.389040 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:34.391772 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.391748 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 15 12:03:34.402462 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.402438 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a77b2b1-873f-4271-aabe-fcdff5647007-metrics-certs\") pod \"network-metrics-daemon-s62bl\" (UID: \"4a77b2b1-873f-4271-aabe-fcdff5647007\") " pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:34.439555 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.439522 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9l82m\"" Sep 15 12:03:34.447724 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.447692 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62bl" Sep 15 12:03:34.490119 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.490084 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:34.490292 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.490158 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cdr\" (UniqueName: \"kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr\") pod \"network-check-target-9mw48\" (UID: \"e4efc2ad-2afa-4eab-9503-6b52ba8cab77\") " pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:34.493052 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.493023 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 15 12:03:34.493190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.493063 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 15 12:03:34.504055 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.504024 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d410c-5eea-4f7f-a913-f86efa9a74dc-original-pull-secret\") pod \"global-pull-secret-syncer-dktw7\" (UID: \"859d410c-5eea-4f7f-a913-f86efa9a74dc\") " pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:34.504877 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.504858 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 15 12:03:34.514229 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.514203 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9cdr\" (UniqueName: \"kubernetes.io/projected/e4efc2ad-2afa-4eab-9503-6b52ba8cab77-kube-api-access-v9cdr\") pod \"network-check-target-9mw48\" (UID: \"e4efc2ad-2afa-4eab-9503-6b52ba8cab77\") " pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:34.725021 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.724985 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k48ps\"" Sep 15 12:03:34.730625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.730598 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dktw7" Sep 15 12:03:34.732635 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:34.732604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:35.520754 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:35.520575 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9mw48"] Sep 15 12:03:35.552179 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:35.550131 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s62bl"] Sep 15 12:03:35.553754 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:35.553423 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dktw7"] Sep 15 12:03:35.664798 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:35.664759 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4efc2ad_2afa_4eab_9503_6b52ba8cab77.slice/crio-a852e0976372ddc13eeb0fe2588faf04b4af2dda07f2a0883cab0e2ac30c35e3 WatchSource:0}: Error finding container a852e0976372ddc13eeb0fe2588faf04b4af2dda07f2a0883cab0e2ac30c35e3: Status 404 returned error can't find the container with id a852e0976372ddc13eeb0fe2588faf04b4af2dda07f2a0883cab0e2ac30c35e3 Sep 15 12:03:35.665648 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:35.665628 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a77b2b1_873f_4271_aabe_fcdff5647007.slice/crio-c407e299005e9266c9ea78e51a79bb8def4e0ff41ed1a4ae08e1f49eb7ed18f6 WatchSource:0}: Error finding container c407e299005e9266c9ea78e51a79bb8def4e0ff41ed1a4ae08e1f49eb7ed18f6: Status 404 returned error can't find the container with id c407e299005e9266c9ea78e51a79bb8def4e0ff41ed1a4ae08e1f49eb7ed18f6 Sep 15 12:03:35.666521 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:03:35.666471 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod859d410c_5eea_4f7f_a913_f86efa9a74dc.slice/crio-a7d7b06b52104e612569c152bc6c0d15519426af0764ac95f7a13784ac3804b0 WatchSource:0}: Error finding container a7d7b06b52104e612569c152bc6c0d15519426af0764ac95f7a13784ac3804b0: Status 404 returned error can't find the container with id a7d7b06b52104e612569c152bc6c0d15519426af0764ac95f7a13784ac3804b0 Sep 15 12:03:36.041565 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:36.041518 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerStarted","Data":"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f"} Sep 15 12:03:36.041740 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:36.041620 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerStarted","Data":"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62"} Sep 15 12:03:36.046169 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:36.046083 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerStarted","Data":"707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2"} Sep 15 12:03:36.047516 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:36.047469 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s62bl" event={"ID":"4a77b2b1-873f-4271-aabe-fcdff5647007","Type":"ContainerStarted","Data":"c407e299005e9266c9ea78e51a79bb8def4e0ff41ed1a4ae08e1f49eb7ed18f6"} Sep 15 12:03:36.048779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:36.048752 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dktw7" event={"ID":"859d410c-5eea-4f7f-a913-f86efa9a74dc","Type":"ContainerStarted","Data":"a7d7b06b52104e612569c152bc6c0d15519426af0764ac95f7a13784ac3804b0"} Sep 15 12:03:36.052300 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:36.052272 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9mw48" event={"ID":"e4efc2ad-2afa-4eab-9503-6b52ba8cab77","Type":"ContainerStarted","Data":"a852e0976372ddc13eeb0fe2588faf04b4af2dda07f2a0883cab0e2ac30c35e3"} Sep 15 12:03:36.075043 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:36.074956 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.0595408 podStartE2EDuration="11.074933823s" totalImagesPullingTime="6.993998539s" totalInitContainerRuntime="21.394484ms" isStatefulPod=false podCreationTimestamp="2026-09-15 12:03:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:36.07332201 +0000 UTC m=+67.830427107" watchObservedRunningTime="2026-09-15 12:03:36.074933823 +0000 UTC m=+67.832038896" Sep 15 12:03:38.067286 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:38.067197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerStarted","Data":"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054"} Sep 15 12:03:38.067286 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:38.067279 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerStarted","Data":"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258"} Sep 15 12:03:38.067817 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:38.067325 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerStarted","Data":"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f"} Sep 15 12:03:38.067817 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:38.067365 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerStarted","Data":"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500"} Sep 15 12:03:38.101774 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:38.101712 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.738954666 podStartE2EDuration="8.10169451s" totalImagesPullingTime="5.343643204s" totalInitContainerRuntime="19.09664ms" isStatefulPod=false podCreationTimestamp="2026-09-15 12:03:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:38.099831781 +0000 UTC m=+69.856936900" watchObservedRunningTime="2026-09-15 12:03:38.10169451 +0000 UTC m=+69.858799580" Sep 15 12:03:41.086621 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:41.086582 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:03:41.087231 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:41.087208 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s62bl" event={"ID":"4a77b2b1-873f-4271-aabe-fcdff5647007","Type":"ContainerStarted","Data":"a72eae505fe059c10b5f9b71a40cd87e794834ed708c7269f8ddd7a6ab2fff27"} Sep 15 12:03:41.087300 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:41.087250 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s62bl" event={"ID":"4a77b2b1-873f-4271-aabe-fcdff5647007","Type":"ContainerStarted","Data":"0f37fab224060071c0e11ab875bd8471f5ee86251c60900e23816d63a6dba0ba"} Sep 15 12:03:41.088833 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:41.088804 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dktw7" event={"ID":"859d410c-5eea-4f7f-a913-f86efa9a74dc","Type":"ContainerStarted","Data":"711b6ce028d052a4995b0cb639ef81cfc59c2ae96d08cf9596eb7a6f770980b4"} Sep 15 12:03:41.090335 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:41.090310 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9mw48" event={"ID":"e4efc2ad-2afa-4eab-9503-6b52ba8cab77","Type":"ContainerStarted","Data":"1f8d8e3eb21dc90242eeffef1bced82e430bd608d5f259bd540835823827711e"} Sep 15 12:03:41.090450 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:41.090434 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:03:41.108698 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:41.108647 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s62bl" podStartSLOduration=68.08437177 podStartE2EDuration="1m13.108631204s" totalImagesPullingTime="5.024259434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:41.107674411 +0000 UTC m=+72.864779485" watchObservedRunningTime="2026-09-15 12:03:41.108631204 +0000 UTC m=+72.865736274" Sep 15 12:03:41.137058 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:41.137004 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9mw48" podStartSLOduration=68.129922818 podStartE2EDuration="1m13.136985589s" totalImagesPullingTime="5.007062771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:41.133428243 +0000 UTC m=+72.890533314" watchObservedRunningTime="2026-09-15 12:03:41.136985589 +0000 UTC m=+72.894090660" Sep 15 12:03:43.100974 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:43.100942 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:43.101333 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:43.101222 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:43.106560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:43.106539 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:43.127369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:43.127327 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dktw7" podStartSLOduration=70.104787366 podStartE2EDuration="1m15.127311527s" totalImagesPullingTime="5.022524161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:03:41.163972762 +0000 UTC m=+72.921077835" watchObservedRunningTime="2026-09-15 12:03:43.127311527 +0000 UTC m=+74.884416598" Sep 15 12:03:44.105036 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:44.105006 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:03:44.160986 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:44.160958 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fd7f7dd8f-ntcpv"] Sep 15 12:03:49.274361 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:49.274312 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:49.274774 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:49.274385 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:03:51.284425 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:03:51.284301 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rz5vx_14a320c1-cabe-447c-af4c-84fc10143b41/serve-healthcheck-canary/0.log" Sep 15 12:04:09.181007 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.180946 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fd7f7dd8f-ntcpv" podUID="cffc8507-c99a-4683-91e0-0daf66174291" containerName="console" containerID="cri-o://c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4" gracePeriod=15 Sep 15 12:04:09.279306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.279270 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:04:09.283163 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.283132 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6ccf494cc7-zjzqk" Sep 15 12:04:09.428810 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.428782 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:04:09.499292 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.499255 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-oauth-config\") pod \"cffc8507-c99a-4683-91e0-0daf66174291\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " Sep 15 12:04:09.499292 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.499296 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-console-config\") pod \"cffc8507-c99a-4683-91e0-0daf66174291\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " Sep 15 12:04:09.499569 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.499331 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nhgd\" (UniqueName: \"kubernetes.io/projected/cffc8507-c99a-4683-91e0-0daf66174291-kube-api-access-9nhgd\") pod \"cffc8507-c99a-4683-91e0-0daf66174291\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " Sep 15 12:04:09.499569 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.499391 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-oauth-serving-cert\") pod \"cffc8507-c99a-4683-91e0-0daf66174291\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " Sep 15 12:04:09.499569 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.499414 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-service-ca\") pod \"cffc8507-c99a-4683-91e0-0daf66174291\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " Sep 15 12:04:09.499569 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.499435 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-serving-cert\") pod \"cffc8507-c99a-4683-91e0-0daf66174291\" (UID: \"cffc8507-c99a-4683-91e0-0daf66174291\") " Sep 15 12:04:09.499841 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.499806 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-console-config" (OuterVolumeSpecName: "console-config") pod "cffc8507-c99a-4683-91e0-0daf66174291" (UID: "cffc8507-c99a-4683-91e0-0daf66174291"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:09.499841 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.499826 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cffc8507-c99a-4683-91e0-0daf66174291" (UID: "cffc8507-c99a-4683-91e0-0daf66174291"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:09.499992 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.499814 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-service-ca" (OuterVolumeSpecName: "service-ca") pod "cffc8507-c99a-4683-91e0-0daf66174291" (UID: "cffc8507-c99a-4683-91e0-0daf66174291"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:09.501873 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.501840 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cffc8507-c99a-4683-91e0-0daf66174291" (UID: "cffc8507-c99a-4683-91e0-0daf66174291"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:09.502240 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.502219 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cffc8507-c99a-4683-91e0-0daf66174291" (UID: "cffc8507-c99a-4683-91e0-0daf66174291"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:09.502310 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.502253 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cffc8507-c99a-4683-91e0-0daf66174291-kube-api-access-9nhgd" (OuterVolumeSpecName: "kube-api-access-9nhgd") pod "cffc8507-c99a-4683-91e0-0daf66174291" (UID: "cffc8507-c99a-4683-91e0-0daf66174291"). InnerVolumeSpecName "kube-api-access-9nhgd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:04:09.601050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.601008 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nhgd\" (UniqueName: \"kubernetes.io/projected/cffc8507-c99a-4683-91e0-0daf66174291-kube-api-access-9nhgd\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:09.601050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.601042 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-oauth-serving-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:09.601050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.601053 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:09.601050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.601062 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-serving-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:09.601310 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.601070 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cffc8507-c99a-4683-91e0-0daf66174291-console-oauth-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:09.601310 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:09.601078 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cffc8507-c99a-4683-91e0-0daf66174291-console-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:10.210637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:10.210598 2582 generic.go:356] "Generic (PLEG): container finished" podID="cffc8507-c99a-4683-91e0-0daf66174291" containerID="c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4" exitCode=0 Sep 15 12:04:10.210637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:10.210640 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd7f7dd8f-ntcpv" Sep 15 12:04:10.211119 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:10.210690 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd7f7dd8f-ntcpv" event={"ID":"cffc8507-c99a-4683-91e0-0daf66174291","Type":"ContainerDied","Data":"c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4"} Sep 15 12:04:10.211119 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:10.210740 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd7f7dd8f-ntcpv" event={"ID":"cffc8507-c99a-4683-91e0-0daf66174291","Type":"ContainerDied","Data":"d688258be5ce11666e4b76db74d682f4f67266f32012ba7619e4a042aa1ef528"} Sep 15 12:04:10.211119 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:10.210764 2582 scope.go:117] "RemoveContainer" containerID="c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4" Sep 15 12:04:10.220252 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:10.220230 2582 scope.go:117] "RemoveContainer" containerID="c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4" Sep 15 12:04:10.220621 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:10.220600 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4\": container with ID starting with c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4 not found: ID does not exist" containerID="c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4" Sep 15 12:04:10.220707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:10.220633 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4"} err="failed to get container status \"c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4\": rpc error: code = NotFound desc = could not find container \"c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4\": container with ID starting with c7122a760369e2839c40ee04ab3dbbf6142d78c1a2946379780ad7d77c6af0b4 not found: ID does not exist" Sep 15 12:04:10.233405 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:10.233377 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fd7f7dd8f-ntcpv"] Sep 15 12:04:10.239085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:10.239058 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fd7f7dd8f-ntcpv"] Sep 15 12:04:10.713172 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:10.713134 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cffc8507-c99a-4683-91e0-0daf66174291" path="/var/lib/kubelet/pods/cffc8507-c99a-4683-91e0-0daf66174291/volumes" Sep 15 12:04:12.097317 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:12.097282 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9mw48" Sep 15 12:04:31.087250 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:31.087205 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:31.106914 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:31.106878 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:31.309563 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:31.309535 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:44.795975 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:44.795937 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 12:04:44.796405 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:44.796221 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="alertmanager" containerID="cri-o://23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b" gracePeriod=120 Sep 15 12:04:44.796405 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:44.796298 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy-metric" containerID="cri-o://5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da" gracePeriod=120 Sep 15 12:04:44.796405 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:44.796330 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy" containerID="cri-o://c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d" gracePeriod=120 Sep 15 12:04:44.796405 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:44.796309 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="prom-label-proxy" containerID="cri-o://707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2" gracePeriod=120 Sep 15 12:04:44.796630 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:44.796387 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="config-reloader" containerID="cri-o://6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40" gracePeriod=120 Sep 15 12:04:44.796630 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:44.796356 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy-web" containerID="cri-o://a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c" gracePeriod=120 Sep 15 12:04:45.347287 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:45.347247 2582 generic.go:356] "Generic (PLEG): container finished" podID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerID="707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2" exitCode=0 Sep 15 12:04:45.347287 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:45.347279 2582 generic.go:356] "Generic (PLEG): container finished" podID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerID="5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da" exitCode=0 Sep 15 12:04:45.347287 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:45.347287 2582 generic.go:356] "Generic (PLEG): container finished" podID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerID="c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d" exitCode=0 Sep 15 12:04:45.347287 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:45.347293 2582 generic.go:356] "Generic (PLEG): container finished" podID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerID="6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40" exitCode=0 Sep 15 12:04:45.347585 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:45.347303 2582 generic.go:356] "Generic (PLEG): container finished" podID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerID="23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b" exitCode=0 Sep 15 12:04:45.347585 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:45.347314 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerDied","Data":"707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2"} Sep 15 12:04:45.347585 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:45.347376 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerDied","Data":"5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da"} Sep 15 12:04:45.347585 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:45.347406 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerDied","Data":"c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d"} Sep 15 12:04:45.347585 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:45.347430 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerDied","Data":"6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40"} Sep 15 12:04:45.347585 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:45.347453 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerDied","Data":"23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b"} Sep 15 12:04:46.040216 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.040192 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.194613 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194516 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-metrics-client-ca\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194613 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194566 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-web\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194613 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194591 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-volume\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194620 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-trusted-ca-bundle\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194649 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxwgx\" (UniqueName: \"kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-kube-api-access-qxwgx\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194664 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194688 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-main-db\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194711 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-tls-assets\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194746 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-out\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194770 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194794 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-web-config\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194817 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-cluster-tls-config\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.194895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194853 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-main-tls\") pod \"2c8334ab-f85d-46e0-8e09-57d1d497421b\" (UID: \"2c8334ab-f85d-46e0-8e09-57d1d497421b\") " Sep 15 12:04:46.195553 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.194969 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:46.195553 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.195084 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:46.195553 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.195223 2582 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-metrics-client-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.195553 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.195330 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:04:46.198822 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.198794 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:46.199019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.198992 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-kube-api-access-qxwgx" (OuterVolumeSpecName: "kube-api-access-qxwgx") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "kube-api-access-qxwgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:04:46.199178 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.199151 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:46.199178 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.199165 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:04:46.199359 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.199327 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-volume" (OuterVolumeSpecName: "config-volume") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:46.199466 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.199443 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:46.199622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.199600 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:46.199683 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.199634 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-out" (OuterVolumeSpecName: "config-out") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:04:46.202952 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.202848 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:46.209386 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.209354 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-web-config" (OuterVolumeSpecName: "web-config") pod "2c8334ab-f85d-46e0-8e09-57d1d497421b" (UID: "2c8334ab-f85d-46e0-8e09-57d1d497421b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:46.296307 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296275 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-main-db\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296307 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296305 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-tls-assets\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296307 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296314 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-out\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296323 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296332 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-web-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296341 2582 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-cluster-tls-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296348 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-main-tls\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296357 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296365 2582 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296374 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c8334ab-f85d-46e0-8e09-57d1d497421b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296382 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxwgx\" (UniqueName: \"kubernetes.io/projected/2c8334ab-f85d-46e0-8e09-57d1d497421b-kube-api-access-qxwgx\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.296554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.296391 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2c8334ab-f85d-46e0-8e09-57d1d497421b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:46.360157 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.360120 2582 generic.go:356] "Generic (PLEG): container finished" podID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerID="a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c" exitCode=0 Sep 15 12:04:46.360321 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.360187 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerDied","Data":"a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c"} Sep 15 12:04:46.360321 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.360220 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.360321 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.360291 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2c8334ab-f85d-46e0-8e09-57d1d497421b","Type":"ContainerDied","Data":"c590cd41c73017c3fad06b79558a50f2bebc08dde77954dfdde62cc33fbca6db"} Sep 15 12:04:46.360434 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.360340 2582 scope.go:117] "RemoveContainer" containerID="707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2" Sep 15 12:04:46.368384 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.368280 2582 scope.go:117] "RemoveContainer" containerID="5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da" Sep 15 12:04:46.375299 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.375282 2582 scope.go:117] "RemoveContainer" containerID="c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d" Sep 15 12:04:46.381948 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.381932 2582 scope.go:117] "RemoveContainer" containerID="a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c" Sep 15 12:04:46.389390 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.389364 2582 scope.go:117] "RemoveContainer" containerID="6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40" Sep 15 12:04:46.389858 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.389835 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 12:04:46.396018 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.395996 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 12:04:46.397306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.397282 2582 scope.go:117] "RemoveContainer" containerID="23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b" Sep 15 12:04:46.404222 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.404207 2582 scope.go:117] "RemoveContainer" containerID="f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1" Sep 15 12:04:46.411810 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.411785 2582 scope.go:117] "RemoveContainer" containerID="707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2" Sep 15 12:04:46.412154 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:46.412129 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2\": container with ID starting with 707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2 not found: ID does not exist" containerID="707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2" Sep 15 12:04:46.412247 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.412165 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2"} err="failed to get container status \"707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2\": rpc error: code = NotFound desc = could not find container \"707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2\": container with ID starting with 707622df3fd438852bda8cbb1cfe2dddf1956b874358455182911bce926edfc2 not found: ID does not exist" Sep 15 12:04:46.412247 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.412190 2582 scope.go:117] "RemoveContainer" containerID="5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da" Sep 15 12:04:46.412540 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:46.412518 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da\": container with ID starting with 5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da not found: ID does not exist" containerID="5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da" Sep 15 12:04:46.412623 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.412549 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da"} err="failed to get container status \"5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da\": rpc error: code = NotFound desc = could not find container \"5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da\": container with ID starting with 5143759978a343d09eee83ee67305cab207986bdf03e4bbe384cb6ac0ac922da not found: ID does not exist" Sep 15 12:04:46.412623 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.412566 2582 scope.go:117] "RemoveContainer" containerID="c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d" Sep 15 12:04:46.412855 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:46.412838 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d\": container with ID starting with c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d not found: ID does not exist" containerID="c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d" Sep 15 12:04:46.412900 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.412860 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d"} err="failed to get container status \"c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d\": rpc error: code = NotFound desc = could not find container \"c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d\": container with ID starting with c7ca3325c4e6b939b589567d6752c9dc9f1ad89a7bf896098619e73424e3a95d not found: ID does not exist" Sep 15 12:04:46.412900 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.412876 2582 scope.go:117] "RemoveContainer" containerID="a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c" Sep 15 12:04:46.413128 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:46.413109 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c\": container with ID starting with a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c not found: ID does not exist" containerID="a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c" Sep 15 12:04:46.413169 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.413134 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c"} err="failed to get container status \"a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c\": rpc error: code = NotFound desc = could not find container \"a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c\": container with ID starting with a7f2e4a49fc73b9c9edda02e769aeba9174cb7ae5ac21c4415cfa6c1f337b52c not found: ID does not exist" Sep 15 12:04:46.413169 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.413151 2582 scope.go:117] "RemoveContainer" containerID="6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40" Sep 15 12:04:46.413384 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:46.413370 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40\": container with ID starting with 6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40 not found: ID does not exist" containerID="6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40" Sep 15 12:04:46.413424 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.413386 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40"} err="failed to get container status \"6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40\": rpc error: code = NotFound desc = could not find container \"6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40\": container with ID starting with 6e602950a7d9eb073287a406492a9add8bc0c8a8420bc7def2160ebc2336db40 not found: ID does not exist" Sep 15 12:04:46.413424 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.413399 2582 scope.go:117] "RemoveContainer" containerID="23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b" Sep 15 12:04:46.413681 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:46.413667 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b\": container with ID starting with 23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b not found: ID does not exist" containerID="23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b" Sep 15 12:04:46.413733 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.413683 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b"} err="failed to get container status \"23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b\": rpc error: code = NotFound desc = could not find container \"23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b\": container with ID starting with 23a3a6814948c0825bfc990f215333d985eeceb2d8bab9b588b68045d6e28f8b not found: ID does not exist" Sep 15 12:04:46.413733 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.413694 2582 scope.go:117] "RemoveContainer" containerID="f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1" Sep 15 12:04:46.413941 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:46.413924 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1\": container with ID starting with f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1 not found: ID does not exist" containerID="f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1" Sep 15 12:04:46.413979 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.413950 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1"} err="failed to get container status \"f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1\": rpc error: code = NotFound desc = could not find container \"f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1\": container with ID starting with f2f175ec70ba84c3dadf8d6937ddcce74623ceef307c202cde0c344de1960bb1 not found: ID does not exist" Sep 15 12:04:46.423502 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423462 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 12:04:46.423744 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423731 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="alertmanager" Sep 15 12:04:46.423791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423746 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="alertmanager" Sep 15 12:04:46.423791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423755 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy-metric" Sep 15 12:04:46.423791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423760 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy-metric" Sep 15 12:04:46.423791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423768 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="prom-label-proxy" Sep 15 12:04:46.423791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423773 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="prom-label-proxy" Sep 15 12:04:46.423791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423779 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="init-config-reloader" Sep 15 12:04:46.423791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423784 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="init-config-reloader" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423796 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy-web" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423800 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy-web" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423805 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423810 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423817 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cffc8507-c99a-4683-91e0-0daf66174291" containerName="console" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423822 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cffc8507-c99a-4683-91e0-0daf66174291" containerName="console" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423828 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="config-reloader" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423833 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="config-reloader" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423880 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="config-reloader" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423887 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy-metric" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423893 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="prom-label-proxy" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423899 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423904 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cffc8507-c99a-4683-91e0-0daf66174291" containerName="console" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423909 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="alertmanager" Sep 15 12:04:46.424019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.423913 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" containerName="kube-rbac-proxy-web" Sep 15 12:04:46.427050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.427033 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.429807 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.429784 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 15 12:04:46.430067 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.430019 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 15 12:04:46.430067 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.430038 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-62dhh\"" Sep 15 12:04:46.430067 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.430047 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 15 12:04:46.430246 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.430082 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 15 12:04:46.430385 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.430364 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 15 12:04:46.430524 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.430428 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 15 12:04:46.430524 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.430431 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 15 12:04:46.430524 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.430501 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 15 12:04:46.435190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.435168 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 15 12:04:46.443855 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.443836 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 12:04:46.599348 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599299 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3e584aa-8fdd-4a51-9324-67b86b6d615d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599574 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599362 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a3e584aa-8fdd-4a51-9324-67b86b6d615d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599574 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599414 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3e584aa-8fdd-4a51-9324-67b86b6d615d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599574 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599435 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599574 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599463 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599574 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599526 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-config-volume\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599574 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599543 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt9wq\" (UniqueName: \"kubernetes.io/projected/a3e584aa-8fdd-4a51-9324-67b86b6d615d-kube-api-access-xt9wq\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599844 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599630 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599844 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599674 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599844 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599717 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3e584aa-8fdd-4a51-9324-67b86b6d615d-config-out\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599844 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599748 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-web-config\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599844 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599767 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.599844 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.599823 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3e584aa-8fdd-4a51-9324-67b86b6d615d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700154 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700196 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700430 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700229 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3e584aa-8fdd-4a51-9324-67b86b6d615d-config-out\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700430 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700268 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-web-config\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700430 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700292 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700430 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700322 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3e584aa-8fdd-4a51-9324-67b86b6d615d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700430 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3e584aa-8fdd-4a51-9324-67b86b6d615d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700430 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700384 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a3e584aa-8fdd-4a51-9324-67b86b6d615d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700430 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700412 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3e584aa-8fdd-4a51-9324-67b86b6d615d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700797 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700435 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700797 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700465 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700797 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700527 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-config-volume\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700797 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700553 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt9wq\" (UniqueName: \"kubernetes.io/projected/a3e584aa-8fdd-4a51-9324-67b86b6d615d-kube-api-access-xt9wq\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.700797 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.700772 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a3e584aa-8fdd-4a51-9324-67b86b6d615d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.701195 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.701164 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3e584aa-8fdd-4a51-9324-67b86b6d615d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.701793 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.701725 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3e584aa-8fdd-4a51-9324-67b86b6d615d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.703329 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.703296 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a3e584aa-8fdd-4a51-9324-67b86b6d615d-config-out\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.703551 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.703523 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.703778 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.703755 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-web-config\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.704043 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.703780 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.704043 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.703864 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a3e584aa-8fdd-4a51-9324-67b86b6d615d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.704043 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.703994 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.704043 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.704014 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.704266 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.704230 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-config-volume\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.705424 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.705401 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a3e584aa-8fdd-4a51-9324-67b86b6d615d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.709397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.709376 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c8334ab-f85d-46e0-8e09-57d1d497421b" path="/var/lib/kubelet/pods/2c8334ab-f85d-46e0-8e09-57d1d497421b/volumes" Sep 15 12:04:46.710571 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.710555 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt9wq\" (UniqueName: \"kubernetes.io/projected/a3e584aa-8fdd-4a51-9324-67b86b6d615d-kube-api-access-xt9wq\") pod \"alertmanager-main-0\" (UID: \"a3e584aa-8fdd-4a51-9324-67b86b6d615d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:46.737631 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:46.737604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 12:04:47.014019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:47.013851 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 12:04:47.016048 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:04:47.016020 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e584aa_8fdd_4a51_9324_67b86b6d615d.slice/crio-51663a70d744ad6f5339c8dd8e281432554230748432863ec6da9898da531348 WatchSource:0}: Error finding container 51663a70d744ad6f5339c8dd8e281432554230748432863ec6da9898da531348: Status 404 returned error can't find the container with id 51663a70d744ad6f5339c8dd8e281432554230748432863ec6da9898da531348 Sep 15 12:04:47.365930 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:47.365889 2582 generic.go:356] "Generic (PLEG): container finished" podID="a3e584aa-8fdd-4a51-9324-67b86b6d615d" containerID="8216dbfd17f7a78a460cc2aabce329399d4442848f693044a3a89ffd3654f33c" exitCode=0 Sep 15 12:04:47.366322 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:47.365976 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3e584aa-8fdd-4a51-9324-67b86b6d615d","Type":"ContainerDied","Data":"8216dbfd17f7a78a460cc2aabce329399d4442848f693044a3a89ffd3654f33c"} Sep 15 12:04:47.366322 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:47.366047 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3e584aa-8fdd-4a51-9324-67b86b6d615d","Type":"ContainerStarted","Data":"51663a70d744ad6f5339c8dd8e281432554230748432863ec6da9898da531348"} Sep 15 12:04:48.373276 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:48.373236 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3e584aa-8fdd-4a51-9324-67b86b6d615d","Type":"ContainerStarted","Data":"3f417ca1c9c6e1fb5b2b8cd55aa2802591caf23bb13e8cbac60a66b0941a20f6"} Sep 15 12:04:48.373778 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:48.373355 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3e584aa-8fdd-4a51-9324-67b86b6d615d","Type":"ContainerStarted","Data":"d697321cef7b0ea30fd8fe03fd7e13b240399e0bffeabbd23fe97dcc1c30ba6e"} Sep 15 12:04:48.373778 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:48.373402 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3e584aa-8fdd-4a51-9324-67b86b6d615d","Type":"ContainerStarted","Data":"f7530ceef0c67bb9cc3f1c921e02a209574bd997070866fc4b9d1262352e45cd"} Sep 15 12:04:48.373778 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:48.373444 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3e584aa-8fdd-4a51-9324-67b86b6d615d","Type":"ContainerStarted","Data":"ef9d9a9e1a3795705a86140cf708456fadc8148cbe17e5be34ae3ee8014e5e4f"} Sep 15 12:04:48.373778 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:48.373479 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3e584aa-8fdd-4a51-9324-67b86b6d615d","Type":"ContainerStarted","Data":"bda673bdbf406166aba8e706e274d25b75698f12c6aebedd963f8c884b7fc7f0"} Sep 15 12:04:48.373778 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:48.373547 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a3e584aa-8fdd-4a51-9324-67b86b6d615d","Type":"ContainerStarted","Data":"41f3485a9bb3182707adc582b7b7ca6483b02ad007ee1be456651a334ee20881"} Sep 15 12:04:48.401289 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:48.401194 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.384718725 podStartE2EDuration="2.401173858s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.455133ms" isStatefulPod=false podCreationTimestamp="2026-09-15 12:04:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:04:48.399066513 +0000 UTC m=+140.156171596" watchObservedRunningTime="2026-09-15 12:04:48.401173858 +0000 UTC m=+140.158278930" Sep 15 12:04:49.082843 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.082809 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 12:04:49.083127 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.083083 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="prometheus" containerID="cri-o://54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62" gracePeriod=600 Sep 15 12:04:49.083215 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.083125 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="thanos-sidecar" containerID="cri-o://a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500" gracePeriod=600 Sep 15 12:04:49.083215 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.083148 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054" gracePeriod=600 Sep 15 12:04:49.083215 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.083104 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy" containerID="cri-o://eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258" gracePeriod=600 Sep 15 12:04:49.083380 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.083134 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="config-reloader" containerID="cri-o://5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f" gracePeriod=600 Sep 15 12:04:49.083380 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.083348 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy-web" containerID="cri-o://c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f" gracePeriod=600 Sep 15 12:04:49.344453 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.344390 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.382736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382701 2582 generic.go:356] "Generic (PLEG): container finished" podID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerID="eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054" exitCode=0 Sep 15 12:04:49.382736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382724 2582 generic.go:356] "Generic (PLEG): container finished" podID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerID="eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258" exitCode=0 Sep 15 12:04:49.382736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382731 2582 generic.go:356] "Generic (PLEG): container finished" podID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerID="c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f" exitCode=0 Sep 15 12:04:49.382736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382737 2582 generic.go:356] "Generic (PLEG): container finished" podID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerID="a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500" exitCode=0 Sep 15 12:04:49.382736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382742 2582 generic.go:356] "Generic (PLEG): container finished" podID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerID="5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f" exitCode=0 Sep 15 12:04:49.382736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382747 2582 generic.go:356] "Generic (PLEG): container finished" podID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerID="54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62" exitCode=0 Sep 15 12:04:49.383397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382777 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerDied","Data":"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054"} Sep 15 12:04:49.383397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382817 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.383397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382852 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerDied","Data":"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258"} Sep 15 12:04:49.383397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382892 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerDied","Data":"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f"} Sep 15 12:04:49.383397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382920 2582 scope.go:117] "RemoveContainer" containerID="eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054" Sep 15 12:04:49.383397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.382923 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerDied","Data":"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500"} Sep 15 12:04:49.383397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.383043 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerDied","Data":"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f"} Sep 15 12:04:49.383397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.383086 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerDied","Data":"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62"} Sep 15 12:04:49.383397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.383127 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"207c7d00-7e67-44f4-bfe8-2e05b94378a6","Type":"ContainerDied","Data":"4b427e3bc7ee8d5afd1c84b9dde1d1cd6b6644bedbc857af883c7485a55d8047"} Sep 15 12:04:49.393818 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.393790 2582 scope.go:117] "RemoveContainer" containerID="eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258" Sep 15 12:04:49.406145 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.405320 2582 scope.go:117] "RemoveContainer" containerID="c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f" Sep 15 12:04:49.413712 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.413690 2582 scope.go:117] "RemoveContainer" containerID="a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500" Sep 15 12:04:49.421071 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.421050 2582 scope.go:117] "RemoveContainer" containerID="5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f" Sep 15 12:04:49.429344 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.429323 2582 scope.go:117] "RemoveContainer" containerID="54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62" Sep 15 12:04:49.437561 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.437542 2582 scope.go:117] "RemoveContainer" containerID="fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555" Sep 15 12:04:49.444614 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.444595 2582 scope.go:117] "RemoveContainer" containerID="eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054" Sep 15 12:04:49.446972 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:49.445602 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": container with ID starting with eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054 not found: ID does not exist" containerID="eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054" Sep 15 12:04:49.446972 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.445754 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054"} err="failed to get container status \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": rpc error: code = NotFound desc = could not find container \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": container with ID starting with eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054 not found: ID does not exist" Sep 15 12:04:49.446972 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.445791 2582 scope.go:117] "RemoveContainer" containerID="eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258" Sep 15 12:04:49.450728 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:49.450697 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": container with ID starting with eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258 not found: ID does not exist" containerID="eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258" Sep 15 12:04:49.450856 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.450735 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258"} err="failed to get container status \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": rpc error: code = NotFound desc = could not find container \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": container with ID starting with eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258 not found: ID does not exist" Sep 15 12:04:49.450856 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.450752 2582 scope.go:117] "RemoveContainer" containerID="c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f" Sep 15 12:04:49.451062 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:49.451045 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": container with ID starting with c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f not found: ID does not exist" containerID="c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f" Sep 15 12:04:49.451121 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.451066 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f"} err="failed to get container status \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": rpc error: code = NotFound desc = could not find container \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": container with ID starting with c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f not found: ID does not exist" Sep 15 12:04:49.451121 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.451079 2582 scope.go:117] "RemoveContainer" containerID="a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500" Sep 15 12:04:49.451352 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:49.451330 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": container with ID starting with a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500 not found: ID does not exist" containerID="a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500" Sep 15 12:04:49.451435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.451359 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500"} err="failed to get container status \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": rpc error: code = NotFound desc = could not find container \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": container with ID starting with a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500 not found: ID does not exist" Sep 15 12:04:49.451435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.451378 2582 scope.go:117] "RemoveContainer" containerID="5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f" Sep 15 12:04:49.451708 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:49.451690 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": container with ID starting with 5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f not found: ID does not exist" containerID="5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f" Sep 15 12:04:49.451779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.451712 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f"} err="failed to get container status \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": rpc error: code = NotFound desc = could not find container \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": container with ID starting with 5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f not found: ID does not exist" Sep 15 12:04:49.451779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.451726 2582 scope.go:117] "RemoveContainer" containerID="54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62" Sep 15 12:04:49.452119 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:49.452068 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": container with ID starting with 54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62 not found: ID does not exist" containerID="54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62" Sep 15 12:04:49.452179 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.452143 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62"} err="failed to get container status \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": rpc error: code = NotFound desc = could not find container \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": container with ID starting with 54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62 not found: ID does not exist" Sep 15 12:04:49.452179 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.452172 2582 scope.go:117] "RemoveContainer" containerID="fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555" Sep 15 12:04:49.452447 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:04:49.452421 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": container with ID starting with fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555 not found: ID does not exist" containerID="fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555" Sep 15 12:04:49.452531 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.452450 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555"} err="failed to get container status \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": rpc error: code = NotFound desc = could not find container \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": container with ID starting with fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555 not found: ID does not exist" Sep 15 12:04:49.452531 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.452468 2582 scope.go:117] "RemoveContainer" containerID="eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054" Sep 15 12:04:49.452731 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.452713 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054"} err="failed to get container status \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": rpc error: code = NotFound desc = could not find container \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": container with ID starting with eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054 not found: ID does not exist" Sep 15 12:04:49.452797 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.452731 2582 scope.go:117] "RemoveContainer" containerID="eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258" Sep 15 12:04:49.452953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.452931 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258"} err="failed to get container status \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": rpc error: code = NotFound desc = could not find container \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": container with ID starting with eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258 not found: ID does not exist" Sep 15 12:04:49.452953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.452953 2582 scope.go:117] "RemoveContainer" containerID="c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f" Sep 15 12:04:49.453211 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.453193 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f"} err="failed to get container status \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": rpc error: code = NotFound desc = could not find container \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": container with ID starting with c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f not found: ID does not exist" Sep 15 12:04:49.453211 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.453210 2582 scope.go:117] "RemoveContainer" containerID="a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500" Sep 15 12:04:49.453422 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.453405 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500"} err="failed to get container status \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": rpc error: code = NotFound desc = could not find container \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": container with ID starting with a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500 not found: ID does not exist" Sep 15 12:04:49.453422 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.453421 2582 scope.go:117] "RemoveContainer" containerID="5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f" Sep 15 12:04:49.453678 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.453661 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f"} err="failed to get container status \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": rpc error: code = NotFound desc = could not find container \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": container with ID starting with 5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f not found: ID does not exist" Sep 15 12:04:49.453678 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.453677 2582 scope.go:117] "RemoveContainer" containerID="54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62" Sep 15 12:04:49.453917 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.453898 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62"} err="failed to get container status \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": rpc error: code = NotFound desc = could not find container \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": container with ID starting with 54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62 not found: ID does not exist" Sep 15 12:04:49.453981 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.453919 2582 scope.go:117] "RemoveContainer" containerID="fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555" Sep 15 12:04:49.454182 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.454164 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555"} err="failed to get container status \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": rpc error: code = NotFound desc = could not find container \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": container with ID starting with fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555 not found: ID does not exist" Sep 15 12:04:49.454262 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.454182 2582 scope.go:117] "RemoveContainer" containerID="eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054" Sep 15 12:04:49.454412 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.454394 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054"} err="failed to get container status \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": rpc error: code = NotFound desc = could not find container \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": container with ID starting with eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054 not found: ID does not exist" Sep 15 12:04:49.454459 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.454414 2582 scope.go:117] "RemoveContainer" containerID="eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258" Sep 15 12:04:49.454683 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.454663 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258"} err="failed to get container status \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": rpc error: code = NotFound desc = could not find container \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": container with ID starting with eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258 not found: ID does not exist" Sep 15 12:04:49.454683 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.454682 2582 scope.go:117] "RemoveContainer" containerID="c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f" Sep 15 12:04:49.454925 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.454909 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f"} err="failed to get container status \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": rpc error: code = NotFound desc = could not find container \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": container with ID starting with c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f not found: ID does not exist" Sep 15 12:04:49.454977 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.454926 2582 scope.go:117] "RemoveContainer" containerID="a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500" Sep 15 12:04:49.455134 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.455118 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500"} err="failed to get container status \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": rpc error: code = NotFound desc = could not find container \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": container with ID starting with a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500 not found: ID does not exist" Sep 15 12:04:49.455178 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.455135 2582 scope.go:117] "RemoveContainer" containerID="5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f" Sep 15 12:04:49.455336 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.455317 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f"} err="failed to get container status \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": rpc error: code = NotFound desc = could not find container \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": container with ID starting with 5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f not found: ID does not exist" Sep 15 12:04:49.455407 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.455337 2582 scope.go:117] "RemoveContainer" containerID="54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62" Sep 15 12:04:49.455596 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.455578 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62"} err="failed to get container status \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": rpc error: code = NotFound desc = could not find container \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": container with ID starting with 54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62 not found: ID does not exist" Sep 15 12:04:49.455596 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.455597 2582 scope.go:117] "RemoveContainer" containerID="fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555" Sep 15 12:04:49.455829 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.455812 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555"} err="failed to get container status \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": rpc error: code = NotFound desc = could not find container \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": container with ID starting with fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555 not found: ID does not exist" Sep 15 12:04:49.455872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.455830 2582 scope.go:117] "RemoveContainer" containerID="eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054" Sep 15 12:04:49.456083 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.456066 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054"} err="failed to get container status \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": rpc error: code = NotFound desc = could not find container \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": container with ID starting with eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054 not found: ID does not exist" Sep 15 12:04:49.456149 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.456085 2582 scope.go:117] "RemoveContainer" containerID="eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258" Sep 15 12:04:49.456317 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.456302 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258"} err="failed to get container status \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": rpc error: code = NotFound desc = could not find container \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": container with ID starting with eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258 not found: ID does not exist" Sep 15 12:04:49.456370 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.456317 2582 scope.go:117] "RemoveContainer" containerID="c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f" Sep 15 12:04:49.456555 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.456538 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f"} err="failed to get container status \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": rpc error: code = NotFound desc = could not find container \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": container with ID starting with c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f not found: ID does not exist" Sep 15 12:04:49.456604 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.456555 2582 scope.go:117] "RemoveContainer" containerID="a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500" Sep 15 12:04:49.456759 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.456744 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500"} err="failed to get container status \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": rpc error: code = NotFound desc = could not find container \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": container with ID starting with a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500 not found: ID does not exist" Sep 15 12:04:49.456804 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.456759 2582 scope.go:117] "RemoveContainer" containerID="5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f" Sep 15 12:04:49.456998 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.456981 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f"} err="failed to get container status \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": rpc error: code = NotFound desc = could not find container \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": container with ID starting with 5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f not found: ID does not exist" Sep 15 12:04:49.457053 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.456999 2582 scope.go:117] "RemoveContainer" containerID="54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62" Sep 15 12:04:49.457224 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.457200 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62"} err="failed to get container status \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": rpc error: code = NotFound desc = could not find container \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": container with ID starting with 54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62 not found: ID does not exist" Sep 15 12:04:49.457224 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.457222 2582 scope.go:117] "RemoveContainer" containerID="fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555" Sep 15 12:04:49.457510 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.457476 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555"} err="failed to get container status \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": rpc error: code = NotFound desc = could not find container \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": container with ID starting with fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555 not found: ID does not exist" Sep 15 12:04:49.457554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.457511 2582 scope.go:117] "RemoveContainer" containerID="eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054" Sep 15 12:04:49.457726 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.457709 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054"} err="failed to get container status \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": rpc error: code = NotFound desc = could not find container \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": container with ID starting with eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054 not found: ID does not exist" Sep 15 12:04:49.457772 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.457726 2582 scope.go:117] "RemoveContainer" containerID="eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258" Sep 15 12:04:49.457952 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.457925 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258"} err="failed to get container status \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": rpc error: code = NotFound desc = could not find container \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": container with ID starting with eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258 not found: ID does not exist" Sep 15 12:04:49.457952 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.457948 2582 scope.go:117] "RemoveContainer" containerID="c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f" Sep 15 12:04:49.458191 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.458175 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f"} err="failed to get container status \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": rpc error: code = NotFound desc = could not find container \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": container with ID starting with c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f not found: ID does not exist" Sep 15 12:04:49.458254 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.458192 2582 scope.go:117] "RemoveContainer" containerID="a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500" Sep 15 12:04:49.458407 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.458391 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500"} err="failed to get container status \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": rpc error: code = NotFound desc = could not find container \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": container with ID starting with a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500 not found: ID does not exist" Sep 15 12:04:49.458454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.458408 2582 scope.go:117] "RemoveContainer" containerID="5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f" Sep 15 12:04:49.458677 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.458656 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f"} err="failed to get container status \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": rpc error: code = NotFound desc = could not find container \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": container with ID starting with 5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f not found: ID does not exist" Sep 15 12:04:49.458677 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.458675 2582 scope.go:117] "RemoveContainer" containerID="54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62" Sep 15 12:04:49.458893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.458879 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62"} err="failed to get container status \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": rpc error: code = NotFound desc = could not find container \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": container with ID starting with 54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62 not found: ID does not exist" Sep 15 12:04:49.458893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.458893 2582 scope.go:117] "RemoveContainer" containerID="fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555" Sep 15 12:04:49.459121 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.459101 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555"} err="failed to get container status \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": rpc error: code = NotFound desc = could not find container \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": container with ID starting with fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555 not found: ID does not exist" Sep 15 12:04:49.459121 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.459118 2582 scope.go:117] "RemoveContainer" containerID="eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054" Sep 15 12:04:49.459342 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.459327 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054"} err="failed to get container status \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": rpc error: code = NotFound desc = could not find container \"eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054\": container with ID starting with eb69a7ce990e1870408a9facac4adce389d6546dafc560ff7df5b3594d987054 not found: ID does not exist" Sep 15 12:04:49.459342 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.459342 2582 scope.go:117] "RemoveContainer" containerID="eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258" Sep 15 12:04:49.459593 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.459571 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258"} err="failed to get container status \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": rpc error: code = NotFound desc = could not find container \"eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258\": container with ID starting with eddae80e4d785ff9fea03e1937afff71623205670461288647a6afc837739258 not found: ID does not exist" Sep 15 12:04:49.459649 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.459596 2582 scope.go:117] "RemoveContainer" containerID="c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f" Sep 15 12:04:49.459786 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.459771 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f"} err="failed to get container status \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": rpc error: code = NotFound desc = could not find container \"c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f\": container with ID starting with c23cb949c1d4237aaaa3e22437bc019d7b6b26e189562762f62f3231714d157f not found: ID does not exist" Sep 15 12:04:49.459839 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.459787 2582 scope.go:117] "RemoveContainer" containerID="a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500" Sep 15 12:04:49.459979 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.459965 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500"} err="failed to get container status \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": rpc error: code = NotFound desc = could not find container \"a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500\": container with ID starting with a7588a6767c2ef985368ebe31835e77e753bde76ad1186fd04be46019257b500 not found: ID does not exist" Sep 15 12:04:49.460027 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.459979 2582 scope.go:117] "RemoveContainer" containerID="5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f" Sep 15 12:04:49.460181 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.460168 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f"} err="failed to get container status \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": rpc error: code = NotFound desc = could not find container \"5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f\": container with ID starting with 5756f7e4a83511560a2ae94888de4e0ada8c51b65d7cf0f9514566c7ad9ea59f not found: ID does not exist" Sep 15 12:04:49.460236 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.460181 2582 scope.go:117] "RemoveContainer" containerID="54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62" Sep 15 12:04:49.460420 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.460400 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62"} err="failed to get container status \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": rpc error: code = NotFound desc = could not find container \"54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62\": container with ID starting with 54db661d697fa0cfd192cb50824ea5557af19c0448909a2a2ddf18d0cebc8d62 not found: ID does not exist" Sep 15 12:04:49.460469 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.460420 2582 scope.go:117] "RemoveContainer" containerID="fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555" Sep 15 12:04:49.460676 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.460659 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555"} err="failed to get container status \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": rpc error: code = NotFound desc = could not find container \"fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555\": container with ID starting with fe79423512b4c095194bde1ab9616e15b929b54225584ba5251e2962c11a2555 not found: ID does not exist" Sep 15 12:04:49.524143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524109 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524156 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-trusted-ca-bundle\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524177 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-tls-assets\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524202 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-thanos-prometheus-http-client-file\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524229 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-grpc-tls\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524259 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-serving-certs-ca-bundle\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524297 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp6qs\" (UniqueName: \"kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-kube-api-access-tp6qs\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524325 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config-out\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524354 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-tls\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524383 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-kubelet-serving-ca-bundle\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524431 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-metrics-client-ca\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524462 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524514 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-1\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524540 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-2\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524571 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-metrics-client-certs\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524602 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-web-config\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524624 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-db\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524653 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524680 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-kube-rbac-proxy\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.524730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524725 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-0\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.525468 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.524877 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:49.525468 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.525000 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:49.525468 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.525041 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config\") pod \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\" (UID: \"207c7d00-7e67-44f4-bfe8-2e05b94378a6\") " Sep 15 12:04:49.525468 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.525338 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:49.525693 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.525522 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.525693 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.525542 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.525693 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.525558 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.525693 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.525571 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.526517 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.526466 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:49.527211 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.527176 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:49.527636 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.527435 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:49.528266 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.528240 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:04:49.528348 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.528275 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:49.528348 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.528274 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:04:49.528722 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.528684 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:04:49.528882 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.528858 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:49.529275 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.529228 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:49.529767 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.529740 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-kube-api-access-tp6qs" (OuterVolumeSpecName: "kube-api-access-tp6qs") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "kube-api-access-tp6qs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:04:49.529767 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.529759 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config" (OuterVolumeSpecName: "config") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:49.530382 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.530355 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:49.530668 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.530646 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config-out" (OuterVolumeSpecName: "config-out") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:04:49.530830 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.530812 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:49.530981 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.530965 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:49.538589 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.538565 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-web-config" (OuterVolumeSpecName: "web-config") pod "207c7d00-7e67-44f4-bfe8-2e05b94378a6" (UID: "207c7d00-7e67-44f4-bfe8-2e05b94378a6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:04:49.626190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626097 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-configmap-metrics-client-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626142 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626153 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626163 2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-metrics-client-certs\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626172 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-web-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626180 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-db\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626188 2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-kube-rbac-proxy\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626195 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/207c7d00-7e67-44f4-bfe8-2e05b94378a6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626204 2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626213 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626222 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-tls-assets\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626230 2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626238 2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-grpc-tls\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626245 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp6qs\" (UniqueName: \"kubernetes.io/projected/207c7d00-7e67-44f4-bfe8-2e05b94378a6-kube-api-access-tp6qs\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626252 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/207c7d00-7e67-44f4-bfe8-2e05b94378a6-config-out\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.626560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.626259 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/207c7d00-7e67-44f4-bfe8-2e05b94378a6-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:04:49.706857 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.706831 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 12:04:49.711917 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.711895 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 12:04:49.737052 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737022 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 12:04:49.737338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737321 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="config-reloader" Sep 15 12:04:49.737384 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737340 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="config-reloader" Sep 15 12:04:49.737384 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737352 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="init-config-reloader" Sep 15 12:04:49.737384 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737358 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="init-config-reloader" Sep 15 12:04:49.737384 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737366 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="thanos-sidecar" Sep 15 12:04:49.737384 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737371 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="thanos-sidecar" Sep 15 12:04:49.737384 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737383 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737388 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737394 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="prometheus" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737402 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="prometheus" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737411 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy-web" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737416 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy-web" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737421 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy-thanos" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737426 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy-thanos" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737463 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737471 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="config-reloader" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737496 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="thanos-sidecar" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737504 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="prometheus" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737509 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy-web" Sep 15 12:04:49.737587 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.737516 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" containerName="kube-rbac-proxy-thanos" Sep 15 12:04:49.741102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.741082 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.744005 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.743985 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 15 12:04:49.744111 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.744004 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 15 12:04:49.744692 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.744676 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 15 12:04:49.744764 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.744740 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 15 12:04:49.744878 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.744860 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 15 12:04:49.745005 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.744898 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 15 12:04:49.745005 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.744944 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 15 12:04:49.745005 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.744988 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 15 12:04:49.745158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.745137 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 15 12:04:49.745377 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.745361 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 15 12:04:49.745377 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.745369 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 15 12:04:49.745499 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.745393 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1in3jebbq3v49\"" Sep 15 12:04:49.746117 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.746102 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-lrpq6\"" Sep 15 12:04:49.746177 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.746106 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 15 12:04:49.752100 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.752078 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 15 12:04:49.757597 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.757571 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 15 12:04:49.771974 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.771948 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 12:04:49.827415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827339 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-config\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827402 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827433 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpxkq\" (UniqueName: \"kubernetes.io/projected/7348ef62-d03a-4a28-9411-9f975a6bab60-kube-api-access-vpxkq\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827814 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827513 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827814 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827539 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827814 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827563 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827814 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827591 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827814 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827624 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827814 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827647 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827814 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827736 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.827814 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827815 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.828763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827864 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.828763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827886 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7348ef62-d03a-4a28-9411-9f975a6bab60-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.828763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827909 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.828763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827930 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-web-config\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.828763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827967 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.828763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.827993 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.828763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.828020 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.828763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.828045 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7348ef62-d03a-4a28-9411-9f975a6bab60-config-out\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.929507 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7348ef62-d03a-4a28-9411-9f975a6bab60-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.929507 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929427 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.929507 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929452 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-web-config\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.929813 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929518 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.929813 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.929813 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929664 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.929813 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7348ef62-d03a-4a28-9411-9f975a6bab60-config-out\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.929813 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929738 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-config\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.929813 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929781 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.929813 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929807 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929830 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpxkq\" (UniqueName: \"kubernetes.io/projected/7348ef62-d03a-4a28-9411-9f975a6bab60-kube-api-access-vpxkq\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929892 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929917 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929941 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.929985 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.930006 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.930037 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.930072 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.930111 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930654 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.930284 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930785 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.930757 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.930989 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.930964 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.932197 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.931746 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.932197 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.931986 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.932197 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.932079 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.932927 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.932623 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.933067 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.933040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7348ef62-d03a-4a28-9411-9f975a6bab60-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.933475 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.933161 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.933475 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.933277 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.933817 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.933790 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7348ef62-d03a-4a28-9411-9f975a6bab60-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.934330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.934296 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.934559 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.934540 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-web-config\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.934937 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.934911 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7348ef62-d03a-4a28-9411-9f975a6bab60-config-out\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.935158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.935134 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.935260 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.935237 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.935513 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.935472 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.935810 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.935794 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.936290 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.936273 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7348ef62-d03a-4a28-9411-9f975a6bab60-config\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:49.941907 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:49.941885 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpxkq\" (UniqueName: \"kubernetes.io/projected/7348ef62-d03a-4a28-9411-9f975a6bab60-kube-api-access-vpxkq\") pod \"prometheus-k8s-0\" (UID: \"7348ef62-d03a-4a28-9411-9f975a6bab60\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:50.051765 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:50.051733 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:04:50.336952 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:50.336688 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 15 12:04:50.339109 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:04:50.339080 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7348ef62_d03a_4a28_9411_9f975a6bab60.slice/crio-23b8451bfe9b1fef8616c475afde4e6a9bc48aece4fcaa1ee800b4bc43e5da7c WatchSource:0}: Error finding container 23b8451bfe9b1fef8616c475afde4e6a9bc48aece4fcaa1ee800b4bc43e5da7c: Status 404 returned error can't find the container with id 23b8451bfe9b1fef8616c475afde4e6a9bc48aece4fcaa1ee800b4bc43e5da7c Sep 15 12:04:50.388203 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:50.388175 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7348ef62-d03a-4a28-9411-9f975a6bab60","Type":"ContainerStarted","Data":"23b8451bfe9b1fef8616c475afde4e6a9bc48aece4fcaa1ee800b4bc43e5da7c"} Sep 15 12:04:50.713055 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:50.713030 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="207c7d00-7e67-44f4-bfe8-2e05b94378a6" path="/var/lib/kubelet/pods/207c7d00-7e67-44f4-bfe8-2e05b94378a6/volumes" Sep 15 12:04:51.395113 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:51.395025 2582 generic.go:356] "Generic (PLEG): container finished" podID="7348ef62-d03a-4a28-9411-9f975a6bab60" containerID="27e2566849d4592da3c9eaf4e929d82938446a5fcf9989634479263a3f8e38ca" exitCode=0 Sep 15 12:04:51.395607 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:51.395118 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7348ef62-d03a-4a28-9411-9f975a6bab60","Type":"ContainerDied","Data":"27e2566849d4592da3c9eaf4e929d82938446a5fcf9989634479263a3f8e38ca"} Sep 15 12:04:52.402413 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.402379 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7348ef62-d03a-4a28-9411-9f975a6bab60","Type":"ContainerStarted","Data":"fa9c935446ca80592963745c30b97a8df36cb607c378b7d9957880f74e0b10b6"} Sep 15 12:04:52.402784 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.402441 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7348ef62-d03a-4a28-9411-9f975a6bab60","Type":"ContainerStarted","Data":"78932c016be13f1c62538b3b70b0da09e104d7992e9b2a2fc40bfae586996bf7"} Sep 15 12:04:52.402784 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.402469 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7348ef62-d03a-4a28-9411-9f975a6bab60","Type":"ContainerStarted","Data":"731436e10644de74eeaf937dac28f187ed553d1f7d412f2fb149674f569e9a6d"} Sep 15 12:04:52.402784 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.402522 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7348ef62-d03a-4a28-9411-9f975a6bab60","Type":"ContainerStarted","Data":"6ec3b8919da58c853c5e54776e56a5414fac2c25c6e361a79259ab2103d76955"} Sep 15 12:04:52.402784 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.402545 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7348ef62-d03a-4a28-9411-9f975a6bab60","Type":"ContainerStarted","Data":"d7a5831713122c36f154ca0267673d6ab1f0ac9e24e9b3f46b956e79cf8526e9"} Sep 15 12:04:52.402784 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.402568 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7348ef62-d03a-4a28-9411-9f975a6bab60","Type":"ContainerStarted","Data":"d1c98c6f01223ba98be0731f247654592da18d0ad7c1e392c77cbfcdb431d25e"} Sep 15 12:04:52.432073 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.432022 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.413237079 podStartE2EDuration="3.431999284s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.762205ms" isStatefulPod=false podCreationTimestamp="2026-09-15 12:04:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:04:52.431404114 +0000 UTC m=+144.188509184" watchObservedRunningTime="2026-09-15 12:04:52.431999284 +0000 UTC m=+144.189104357" Sep 15 12:04:52.654004 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.653923 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fc9d99fb5-pvbhm"] Sep 15 12:04:52.656012 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.655995 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.666804 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.666772 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fc9d99fb5-pvbhm"] Sep 15 12:04:52.753197 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.753153 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-trusted-ca-bundle\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.753373 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.753215 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-oauth-serving-cert\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.753373 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.753244 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-console-config\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.753373 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.753266 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-serving-cert\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.753373 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.753296 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74nbs\" (UniqueName: \"kubernetes.io/projected/972a9ffa-9984-43ff-b602-5217428a473a-kube-api-access-74nbs\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.753373 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.753311 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-service-ca\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.753373 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.753329 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-oauth-config\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.853853 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.853804 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-oauth-serving-cert\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.854054 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.853866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-console-config\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.854054 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.853897 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-serving-cert\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.854054 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.853947 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74nbs\" (UniqueName: \"kubernetes.io/projected/972a9ffa-9984-43ff-b602-5217428a473a-kube-api-access-74nbs\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.854054 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.853968 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-service-ca\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.854054 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.853996 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-oauth-config\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.854054 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.854041 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-trusted-ca-bundle\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.855073 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.855037 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-trusted-ca-bundle\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.855622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.855595 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-oauth-serving-cert\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.856195 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.856165 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-console-config\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.856447 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.856420 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-service-ca\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.858642 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.858611 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-oauth-config\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.858718 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.858620 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-serving-cert\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.865910 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.865882 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74nbs\" (UniqueName: \"kubernetes.io/projected/972a9ffa-9984-43ff-b602-5217428a473a-kube-api-access-74nbs\") pod \"console-fc9d99fb5-pvbhm\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:52.965073 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:52.965039 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:04:53.245120 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:53.244954 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fc9d99fb5-pvbhm"] Sep 15 12:04:53.248080 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:04:53.248052 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod972a9ffa_9984_43ff_b602_5217428a473a.slice/crio-a06d603810c0b0482aef984b302ac716b5fa01ee3bd8e5a4ad865684accbfa53 WatchSource:0}: Error finding container a06d603810c0b0482aef984b302ac716b5fa01ee3bd8e5a4ad865684accbfa53: Status 404 returned error can't find the container with id a06d603810c0b0482aef984b302ac716b5fa01ee3bd8e5a4ad865684accbfa53 Sep 15 12:04:53.408977 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:53.408943 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc9d99fb5-pvbhm" event={"ID":"972a9ffa-9984-43ff-b602-5217428a473a","Type":"ContainerStarted","Data":"3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737"} Sep 15 12:04:53.409394 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:53.408996 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc9d99fb5-pvbhm" event={"ID":"972a9ffa-9984-43ff-b602-5217428a473a","Type":"ContainerStarted","Data":"a06d603810c0b0482aef984b302ac716b5fa01ee3bd8e5a4ad865684accbfa53"} Sep 15 12:04:53.428059 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:53.428000 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fc9d99fb5-pvbhm" podStartSLOduration=1.427984286 podStartE2EDuration="1.427984286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:04:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:04:53.426821681 +0000 UTC m=+145.183926950" watchObservedRunningTime="2026-09-15 12:04:53.427984286 +0000 UTC m=+145.185089356" Sep 15 12:04:55.052855 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:04:55.052799 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:05:02.965469 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:02.965413 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:05:02.966008 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:02.965515 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:05:02.970342 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:02.970323 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:05:03.451631 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:03.451550 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:05:03.491435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:03.491402 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c559c8986-cl7kk"] Sep 15 12:05:28.516408 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.516342 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c559c8986-cl7kk" podUID="390b4401-d041-446c-8277-5da3a61a8ef3" containerName="console" containerID="cri-o://dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0" gracePeriod=15 Sep 15 12:05:28.763505 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.763464 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:05:28.845806 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.845702 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-console-config\") pod \"390b4401-d041-446c-8277-5da3a61a8ef3\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " Sep 15 12:05:28.845806 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.845762 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-oauth-config\") pod \"390b4401-d041-446c-8277-5da3a61a8ef3\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " Sep 15 12:05:28.845806 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.845804 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w2z4\" (UniqueName: \"kubernetes.io/projected/390b4401-d041-446c-8277-5da3a61a8ef3-kube-api-access-2w2z4\") pod \"390b4401-d041-446c-8277-5da3a61a8ef3\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " Sep 15 12:05:28.846081 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.845840 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-trusted-ca-bundle\") pod \"390b4401-d041-446c-8277-5da3a61a8ef3\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " Sep 15 12:05:28.846081 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.845868 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-oauth-serving-cert\") pod \"390b4401-d041-446c-8277-5da3a61a8ef3\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " Sep 15 12:05:28.846081 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.845894 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-serving-cert\") pod \"390b4401-d041-446c-8277-5da3a61a8ef3\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " Sep 15 12:05:28.846081 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.845927 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-service-ca\") pod \"390b4401-d041-446c-8277-5da3a61a8ef3\" (UID: \"390b4401-d041-446c-8277-5da3a61a8ef3\") " Sep 15 12:05:28.846264 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.846203 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-console-config" (OuterVolumeSpecName: "console-config") pod "390b4401-d041-446c-8277-5da3a61a8ef3" (UID: "390b4401-d041-446c-8277-5da3a61a8ef3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:05:28.846320 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.846257 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "390b4401-d041-446c-8277-5da3a61a8ef3" (UID: "390b4401-d041-446c-8277-5da3a61a8ef3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:05:28.846443 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.846328 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "390b4401-d041-446c-8277-5da3a61a8ef3" (UID: "390b4401-d041-446c-8277-5da3a61a8ef3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:05:28.846533 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.846506 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-service-ca" (OuterVolumeSpecName: "service-ca") pod "390b4401-d041-446c-8277-5da3a61a8ef3" (UID: "390b4401-d041-446c-8277-5da3a61a8ef3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:05:28.848503 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.848464 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "390b4401-d041-446c-8277-5da3a61a8ef3" (UID: "390b4401-d041-446c-8277-5da3a61a8ef3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:05:28.848789 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.848722 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "390b4401-d041-446c-8277-5da3a61a8ef3" (UID: "390b4401-d041-446c-8277-5da3a61a8ef3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:05:28.848789 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.848724 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/390b4401-d041-446c-8277-5da3a61a8ef3-kube-api-access-2w2z4" (OuterVolumeSpecName: "kube-api-access-2w2z4") pod "390b4401-d041-446c-8277-5da3a61a8ef3" (UID: "390b4401-d041-446c-8277-5da3a61a8ef3"). InnerVolumeSpecName "kube-api-access-2w2z4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:05:28.946872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.946840 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w2z4\" (UniqueName: \"kubernetes.io/projected/390b4401-d041-446c-8277-5da3a61a8ef3-kube-api-access-2w2z4\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:05:28.946872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.946869 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-trusted-ca-bundle\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:05:28.946872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.946878 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-oauth-serving-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:05:28.947111 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.946886 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-serving-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:05:28.947111 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.946896 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:05:28.947111 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.946905 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/390b4401-d041-446c-8277-5da3a61a8ef3-console-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:05:28.947111 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:28.946912 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/390b4401-d041-446c-8277-5da3a61a8ef3-console-oauth-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:05:29.553397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:29.553364 2582 generic.go:356] "Generic (PLEG): container finished" podID="390b4401-d041-446c-8277-5da3a61a8ef3" containerID="dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0" exitCode=0 Sep 15 12:05:29.553790 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:29.553411 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c559c8986-cl7kk" Sep 15 12:05:29.553790 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:29.553412 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c559c8986-cl7kk" event={"ID":"390b4401-d041-446c-8277-5da3a61a8ef3","Type":"ContainerDied","Data":"dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0"} Sep 15 12:05:29.553790 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:29.553539 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c559c8986-cl7kk" event={"ID":"390b4401-d041-446c-8277-5da3a61a8ef3","Type":"ContainerDied","Data":"dea7604b031cdaafac415a7d509b0483740a0f3e85eb178d62522e77b4fc68ea"} Sep 15 12:05:29.553790 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:29.553567 2582 scope.go:117] "RemoveContainer" containerID="dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0" Sep 15 12:05:29.564273 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:29.564253 2582 scope.go:117] "RemoveContainer" containerID="dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0" Sep 15 12:05:29.564897 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:05:29.564862 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0\": container with ID starting with dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0 not found: ID does not exist" containerID="dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0" Sep 15 12:05:29.565007 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:29.564903 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0"} err="failed to get container status \"dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0\": rpc error: code = NotFound desc = could not find container \"dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0\": container with ID starting with dde01c94ae63dc6cb834f9e7768c76d5e86d58d9bd7a3eb7627cac39ae247ba0 not found: ID does not exist" Sep 15 12:05:29.579503 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:29.579452 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c559c8986-cl7kk"] Sep 15 12:05:29.582680 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:29.582654 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c559c8986-cl7kk"] Sep 15 12:05:30.710378 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:30.710347 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="390b4401-d041-446c-8277-5da3a61a8ef3" path="/var/lib/kubelet/pods/390b4401-d041-446c-8277-5da3a61a8ef3/volumes" Sep 15 12:05:50.052710 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:50.052673 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:05:50.068844 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:50.068818 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:05:50.644411 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:05:50.644384 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 15 12:06:12.854989 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.854954 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d5d9dc4fd-dtk5s"] Sep 15 12:06:12.855423 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.855238 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390b4401-d041-446c-8277-5da3a61a8ef3" containerName="console" Sep 15 12:06:12.855423 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.855249 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="390b4401-d041-446c-8277-5da3a61a8ef3" containerName="console" Sep 15 12:06:12.855423 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.855295 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="390b4401-d041-446c-8277-5da3a61a8ef3" containerName="console" Sep 15 12:06:12.858799 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.858780 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:12.869257 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.869233 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d5d9dc4fd-dtk5s"] Sep 15 12:06:12.917346 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.917314 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-oauth-serving-cert\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:12.917534 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.917355 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-oauth-config\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:12.917534 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.917371 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-serving-cert\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:12.917534 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.917447 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-trusted-ca-bundle\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:12.917534 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.917513 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdk82\" (UniqueName: \"kubernetes.io/projected/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-kube-api-access-xdk82\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:12.917697 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.917538 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-config\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:12.917697 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:12.917567 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-service-ca\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.018354 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.018319 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-oauth-serving-cert\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.018354 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.018361 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-oauth-config\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.018625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.018380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-serving-cert\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.018625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.018409 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-trusted-ca-bundle\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.018625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.018429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdk82\" (UniqueName: \"kubernetes.io/projected/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-kube-api-access-xdk82\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.018625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.018449 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-config\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.018625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.018473 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-service-ca\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.019274 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.019199 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-config\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.019389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.019347 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-service-ca\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.019389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.019365 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-oauth-serving-cert\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.019651 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.019627 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-trusted-ca-bundle\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.021143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.021115 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-oauth-config\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.021229 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.021203 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-serving-cert\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.028418 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.028396 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdk82\" (UniqueName: \"kubernetes.io/projected/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-kube-api-access-xdk82\") pod \"console-7d5d9dc4fd-dtk5s\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.168817 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.168734 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:13.442263 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.442195 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d5d9dc4fd-dtk5s"] Sep 15 12:06:13.444405 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:06:13.444375 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d0e40fe_9af9_49e0_babd_8f6b67bfe050.slice/crio-16983096a6c4493abf3465c5d2c5879353036bfdbf8d95ffaf652132d72958ff WatchSource:0}: Error finding container 16983096a6c4493abf3465c5d2c5879353036bfdbf8d95ffaf652132d72958ff: Status 404 returned error can't find the container with id 16983096a6c4493abf3465c5d2c5879353036bfdbf8d95ffaf652132d72958ff Sep 15 12:06:13.711664 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.711574 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5d9dc4fd-dtk5s" event={"ID":"5d0e40fe-9af9-49e0-babd-8f6b67bfe050","Type":"ContainerStarted","Data":"b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc"} Sep 15 12:06:13.711664 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.711634 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5d9dc4fd-dtk5s" event={"ID":"5d0e40fe-9af9-49e0-babd-8f6b67bfe050","Type":"ContainerStarted","Data":"16983096a6c4493abf3465c5d2c5879353036bfdbf8d95ffaf652132d72958ff"} Sep 15 12:06:13.729037 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:13.728984 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d5d9dc4fd-dtk5s" podStartSLOduration=1.7289622850000002 podStartE2EDuration="1.728962285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:06:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:06:13.727984551 +0000 UTC m=+225.485089624" watchObservedRunningTime="2026-09-15 12:06:13.728962285 +0000 UTC m=+225.486067355" Sep 15 12:06:23.169886 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:23.169838 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:23.170474 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:23.169942 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:23.174673 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:23.174650 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:23.751750 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:23.751721 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:06:23.798737 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:23.798698 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fc9d99fb5-pvbhm"] Sep 15 12:06:48.824120 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:48.824081 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fc9d99fb5-pvbhm" podUID="972a9ffa-9984-43ff-b602-5217428a473a" containerName="console" containerID="cri-o://3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737" gracePeriod=15 Sep 15 12:06:49.074364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.074290 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:06:49.117377 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.117347 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-serving-cert\") pod \"972a9ffa-9984-43ff-b602-5217428a473a\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " Sep 15 12:06:49.117570 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.117381 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-oauth-serving-cert\") pod \"972a9ffa-9984-43ff-b602-5217428a473a\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " Sep 15 12:06:49.117570 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.117426 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-service-ca\") pod \"972a9ffa-9984-43ff-b602-5217428a473a\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " Sep 15 12:06:49.117570 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.117453 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-console-config\") pod \"972a9ffa-9984-43ff-b602-5217428a473a\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " Sep 15 12:06:49.117570 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.117498 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74nbs\" (UniqueName: \"kubernetes.io/projected/972a9ffa-9984-43ff-b602-5217428a473a-kube-api-access-74nbs\") pod \"972a9ffa-9984-43ff-b602-5217428a473a\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " Sep 15 12:06:49.117570 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.117544 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-oauth-config\") pod \"972a9ffa-9984-43ff-b602-5217428a473a\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " Sep 15 12:06:49.117834 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.117577 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-trusted-ca-bundle\") pod \"972a9ffa-9984-43ff-b602-5217428a473a\" (UID: \"972a9ffa-9984-43ff-b602-5217428a473a\") " Sep 15 12:06:49.117954 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.117929 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-service-ca" (OuterVolumeSpecName: "service-ca") pod "972a9ffa-9984-43ff-b602-5217428a473a" (UID: "972a9ffa-9984-43ff-b602-5217428a473a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:06:49.118008 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.117966 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-console-config" (OuterVolumeSpecName: "console-config") pod "972a9ffa-9984-43ff-b602-5217428a473a" (UID: "972a9ffa-9984-43ff-b602-5217428a473a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:06:49.118008 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.117975 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "972a9ffa-9984-43ff-b602-5217428a473a" (UID: "972a9ffa-9984-43ff-b602-5217428a473a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:06:49.118075 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.118050 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "972a9ffa-9984-43ff-b602-5217428a473a" (UID: "972a9ffa-9984-43ff-b602-5217428a473a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:06:49.119999 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.119967 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "972a9ffa-9984-43ff-b602-5217428a473a" (UID: "972a9ffa-9984-43ff-b602-5217428a473a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:06:49.120289 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.120257 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "972a9ffa-9984-43ff-b602-5217428a473a" (UID: "972a9ffa-9984-43ff-b602-5217428a473a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:06:49.120289 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.120270 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/972a9ffa-9984-43ff-b602-5217428a473a-kube-api-access-74nbs" (OuterVolumeSpecName: "kube-api-access-74nbs") pod "972a9ffa-9984-43ff-b602-5217428a473a" (UID: "972a9ffa-9984-43ff-b602-5217428a473a"). InnerVolumeSpecName "kube-api-access-74nbs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:06:49.218770 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.218731 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-console-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:06:49.218770 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.218761 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74nbs\" (UniqueName: \"kubernetes.io/projected/972a9ffa-9984-43ff-b602-5217428a473a-kube-api-access-74nbs\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:06:49.218770 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.218774 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-oauth-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:06:49.219028 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.218783 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-trusted-ca-bundle\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:06:49.219028 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.218792 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/972a9ffa-9984-43ff-b602-5217428a473a-console-serving-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:06:49.219028 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.218801 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-oauth-serving-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:06:49.219028 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.218809 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/972a9ffa-9984-43ff-b602-5217428a473a-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:06:49.848020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.847938 2582 generic.go:356] "Generic (PLEG): container finished" podID="972a9ffa-9984-43ff-b602-5217428a473a" containerID="3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737" exitCode=0 Sep 15 12:06:49.848020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.847990 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc9d99fb5-pvbhm" Sep 15 12:06:49.848020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.847995 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc9d99fb5-pvbhm" event={"ID":"972a9ffa-9984-43ff-b602-5217428a473a","Type":"ContainerDied","Data":"3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737"} Sep 15 12:06:49.848591 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.848036 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc9d99fb5-pvbhm" event={"ID":"972a9ffa-9984-43ff-b602-5217428a473a","Type":"ContainerDied","Data":"a06d603810c0b0482aef984b302ac716b5fa01ee3bd8e5a4ad865684accbfa53"} Sep 15 12:06:49.848591 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.848056 2582 scope.go:117] "RemoveContainer" containerID="3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737" Sep 15 12:06:49.856546 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.856519 2582 scope.go:117] "RemoveContainer" containerID="3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737" Sep 15 12:06:49.856858 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:06:49.856840 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737\": container with ID starting with 3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737 not found: ID does not exist" containerID="3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737" Sep 15 12:06:49.856904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.856867 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737"} err="failed to get container status \"3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737\": rpc error: code = NotFound desc = could not find container \"3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737\": container with ID starting with 3eb475ad10216a2c99939a391220b8ff8c78148a2b15e22d6dbea568164b4737 not found: ID does not exist" Sep 15 12:06:49.867870 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.867841 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fc9d99fb5-pvbhm"] Sep 15 12:06:49.870148 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:49.870128 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fc9d99fb5-pvbhm"] Sep 15 12:06:50.710393 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:06:50.710363 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="972a9ffa-9984-43ff-b602-5217428a473a" path="/var/lib/kubelet/pods/972a9ffa-9984-43ff-b602-5217428a473a/volumes" Sep 15 12:07:28.580206 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:28.580180 2582 kubelet.go:1648] "Image garbage collection succeeded" Sep 15 12:07:53.324650 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.324607 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w7hgt"] Sep 15 12:07:53.325106 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.324898 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="972a9ffa-9984-43ff-b602-5217428a473a" containerName="console" Sep 15 12:07:53.325106 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.324908 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="972a9ffa-9984-43ff-b602-5217428a473a" containerName="console" Sep 15 12:07:53.325106 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.324956 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="972a9ffa-9984-43ff-b602-5217428a473a" containerName="console" Sep 15 12:07:53.327705 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.327691 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-w7hgt" Sep 15 12:07:53.330518 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.330472 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-lz2pr\"" Sep 15 12:07:53.330668 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.330518 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 15 12:07:53.331715 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.331696 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 15 12:07:53.341927 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.341900 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w7hgt"] Sep 15 12:07:53.437214 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.437174 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fa71823-ad87-42d0-9fc8-ddf0fe1226ff-bound-sa-token\") pod \"cert-manager-55b4d954d5-w7hgt\" (UID: \"7fa71823-ad87-42d0-9fc8-ddf0fe1226ff\") " pod="cert-manager/cert-manager-55b4d954d5-w7hgt" Sep 15 12:07:53.437391 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.437227 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bcvc\" (UniqueName: \"kubernetes.io/projected/7fa71823-ad87-42d0-9fc8-ddf0fe1226ff-kube-api-access-2bcvc\") pod \"cert-manager-55b4d954d5-w7hgt\" (UID: \"7fa71823-ad87-42d0-9fc8-ddf0fe1226ff\") " pod="cert-manager/cert-manager-55b4d954d5-w7hgt" Sep 15 12:07:53.538645 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.538610 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fa71823-ad87-42d0-9fc8-ddf0fe1226ff-bound-sa-token\") pod \"cert-manager-55b4d954d5-w7hgt\" (UID: \"7fa71823-ad87-42d0-9fc8-ddf0fe1226ff\") " pod="cert-manager/cert-manager-55b4d954d5-w7hgt" Sep 15 12:07:53.538827 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.538661 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bcvc\" (UniqueName: \"kubernetes.io/projected/7fa71823-ad87-42d0-9fc8-ddf0fe1226ff-kube-api-access-2bcvc\") pod \"cert-manager-55b4d954d5-w7hgt\" (UID: \"7fa71823-ad87-42d0-9fc8-ddf0fe1226ff\") " pod="cert-manager/cert-manager-55b4d954d5-w7hgt" Sep 15 12:07:53.547294 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.547262 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fa71823-ad87-42d0-9fc8-ddf0fe1226ff-bound-sa-token\") pod \"cert-manager-55b4d954d5-w7hgt\" (UID: \"7fa71823-ad87-42d0-9fc8-ddf0fe1226ff\") " pod="cert-manager/cert-manager-55b4d954d5-w7hgt" Sep 15 12:07:53.547294 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.547275 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bcvc\" (UniqueName: \"kubernetes.io/projected/7fa71823-ad87-42d0-9fc8-ddf0fe1226ff-kube-api-access-2bcvc\") pod \"cert-manager-55b4d954d5-w7hgt\" (UID: \"7fa71823-ad87-42d0-9fc8-ddf0fe1226ff\") " pod="cert-manager/cert-manager-55b4d954d5-w7hgt" Sep 15 12:07:53.647618 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.647524 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-w7hgt" Sep 15 12:07:53.917126 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.917101 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w7hgt"] Sep 15 12:07:53.919707 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:07:53.919679 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fa71823_ad87_42d0_9fc8_ddf0fe1226ff.slice/crio-e19224af072313c16a8953cd9dca85cc66e9ddeb49b5140c486c28fc9af64db4 WatchSource:0}: Error finding container e19224af072313c16a8953cd9dca85cc66e9ddeb49b5140c486c28fc9af64db4: Status 404 returned error can't find the container with id e19224af072313c16a8953cd9dca85cc66e9ddeb49b5140c486c28fc9af64db4 Sep 15 12:07:53.923926 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:53.923905 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 12:07:54.085501 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:54.085446 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-w7hgt" event={"ID":"7fa71823-ad87-42d0-9fc8-ddf0fe1226ff","Type":"ContainerStarted","Data":"e19224af072313c16a8953cd9dca85cc66e9ddeb49b5140c486c28fc9af64db4"} Sep 15 12:07:55.348715 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.348675 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj"] Sep 15 12:07:55.357158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.356962 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" Sep 15 12:07:55.360031 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.360006 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj"] Sep 15 12:07:55.360179 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.360049 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-6dmwv\"" Sep 15 12:07:55.360179 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.360138 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 15 12:07:55.361426 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.361229 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 15 12:07:55.457778 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.457745 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwq4s\" (UniqueName: \"kubernetes.io/projected/4c92c9b8-295a-45be-af60-2ec60b716fcd-kube-api-access-gwq4s\") pod \"openshift-lws-operator-fd8ccff4c-mhbtj\" (UID: \"4c92c9b8-295a-45be-af60-2ec60b716fcd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" Sep 15 12:07:55.458017 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.457883 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c92c9b8-295a-45be-af60-2ec60b716fcd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mhbtj\" (UID: \"4c92c9b8-295a-45be-af60-2ec60b716fcd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" Sep 15 12:07:55.558472 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.558432 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c92c9b8-295a-45be-af60-2ec60b716fcd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mhbtj\" (UID: \"4c92c9b8-295a-45be-af60-2ec60b716fcd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" Sep 15 12:07:55.558678 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.558558 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwq4s\" (UniqueName: \"kubernetes.io/projected/4c92c9b8-295a-45be-af60-2ec60b716fcd-kube-api-access-gwq4s\") pod \"openshift-lws-operator-fd8ccff4c-mhbtj\" (UID: \"4c92c9b8-295a-45be-af60-2ec60b716fcd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" Sep 15 12:07:55.558911 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.558862 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c92c9b8-295a-45be-af60-2ec60b716fcd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mhbtj\" (UID: \"4c92c9b8-295a-45be-af60-2ec60b716fcd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" Sep 15 12:07:55.567909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.567860 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwq4s\" (UniqueName: \"kubernetes.io/projected/4c92c9b8-295a-45be-af60-2ec60b716fcd-kube-api-access-gwq4s\") pod \"openshift-lws-operator-fd8ccff4c-mhbtj\" (UID: \"4c92c9b8-295a-45be-af60-2ec60b716fcd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" Sep 15 12:07:55.670923 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:55.670850 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" Sep 15 12:07:57.076558 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:57.076435 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj"] Sep 15 12:07:57.079431 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:07:57.079403 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c92c9b8_295a_45be_af60_2ec60b716fcd.slice/crio-60375da5ceeaebf7281d31688ab8cdca045d7114ae982012619d6afc2e2d05a8 WatchSource:0}: Error finding container 60375da5ceeaebf7281d31688ab8cdca045d7114ae982012619d6afc2e2d05a8: Status 404 returned error can't find the container with id 60375da5ceeaebf7281d31688ab8cdca045d7114ae982012619d6afc2e2d05a8 Sep 15 12:07:57.104597 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:57.104566 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" event={"ID":"4c92c9b8-295a-45be-af60-2ec60b716fcd","Type":"ContainerStarted","Data":"60375da5ceeaebf7281d31688ab8cdca045d7114ae982012619d6afc2e2d05a8"} Sep 15 12:07:57.106007 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:57.105981 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-w7hgt" event={"ID":"7fa71823-ad87-42d0-9fc8-ddf0fe1226ff","Type":"ContainerStarted","Data":"3e8ae5be0d236f1fc0d503ef43952ac3d14a8977a0782bf9c9e8f18edabc1b0d"} Sep 15 12:07:57.125763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:07:57.125721 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-w7hgt" podStartSLOduration=1.168170884 podStartE2EDuration="4.125706891s" totalImagesPullingTime="2.957536007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:07:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:07:57.124727316 +0000 UTC m=+328.881832389" watchObservedRunningTime="2026-09-15 12:07:57.125706891 +0000 UTC m=+328.882811963" Sep 15 12:08:00.126446 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:00.126401 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" event={"ID":"4c92c9b8-295a-45be-af60-2ec60b716fcd","Type":"ContainerStarted","Data":"c3a2b8f115f2124020a190b2f9d439ea8c484ea43ea783e5d03f4ec140f1e5c1"} Sep 15 12:08:00.145844 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:00.145787 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhbtj" podStartSLOduration=2.657872257 podStartE2EDuration="5.145771611s" totalImagesPullingTime="2.487899354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:08:00.143838518 +0000 UTC m=+331.900943588" watchObservedRunningTime="2026-09-15 12:08:00.145771611 +0000 UTC m=+331.902876681" Sep 15 12:08:16.216097 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.216066 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg"] Sep 15 12:08:16.219849 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.219826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.224013 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.223985 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-khcjr\"" Sep 15 12:08:16.224013 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.224004 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 15 12:08:16.224193 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.224070 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 15 12:08:16.224350 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.224333 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 15 12:08:16.224390 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.224322 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 15 12:08:16.234993 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.234970 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg"] Sep 15 12:08:16.345768 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.345733 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s24c\" (UniqueName: \"kubernetes.io/projected/4dc67243-5de9-4de1-bc50-df52da5facf8-kube-api-access-8s24c\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.345768 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.345776 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4dc67243-5de9-4de1-bc50-df52da5facf8-manifests\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.346001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.345818 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4dc67243-5de9-4de1-bc50-df52da5facf8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.346001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.345843 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4dc67243-5de9-4de1-bc50-df52da5facf8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.346001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.345861 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dc67243-5de9-4de1-bc50-df52da5facf8-tmp\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.446462 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.446420 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4dc67243-5de9-4de1-bc50-df52da5facf8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.446462 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.446459 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4dc67243-5de9-4de1-bc50-df52da5facf8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.446707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.446476 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dc67243-5de9-4de1-bc50-df52da5facf8-tmp\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.446707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.446669 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s24c\" (UniqueName: \"kubernetes.io/projected/4dc67243-5de9-4de1-bc50-df52da5facf8-kube-api-access-8s24c\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.446787 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.446710 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4dc67243-5de9-4de1-bc50-df52da5facf8-manifests\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.447059 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.447025 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4dc67243-5de9-4de1-bc50-df52da5facf8-manifests\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.449007 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.448985 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dc67243-5de9-4de1-bc50-df52da5facf8-tmp\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.449106 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.449030 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4dc67243-5de9-4de1-bc50-df52da5facf8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.449146 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.449113 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4dc67243-5de9-4de1-bc50-df52da5facf8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.457185 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.457163 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s24c\" (UniqueName: \"kubernetes.io/projected/4dc67243-5de9-4de1-bc50-df52da5facf8-kube-api-access-8s24c\") pod \"opendatahub-operator-controller-manager-7b747b9585-74lgg\" (UID: \"4dc67243-5de9-4de1-bc50-df52da5facf8\") " pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.530342 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.530239 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:16.812034 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:16.812008 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg"] Sep 15 12:08:16.814525 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:08:16.814497 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc67243_5de9_4de1_bc50_df52da5facf8.slice/crio-beed433c516835c9e00e065b2e3db3cf62ff0aa5f0783ffd62c445e3479def18 WatchSource:0}: Error finding container beed433c516835c9e00e065b2e3db3cf62ff0aa5f0783ffd62c445e3479def18: Status 404 returned error can't find the container with id beed433c516835c9e00e065b2e3db3cf62ff0aa5f0783ffd62c445e3479def18 Sep 15 12:08:17.194398 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:17.194306 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" event={"ID":"4dc67243-5de9-4de1-bc50-df52da5facf8","Type":"ContainerStarted","Data":"beed433c516835c9e00e065b2e3db3cf62ff0aa5f0783ffd62c445e3479def18"} Sep 15 12:08:20.211339 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:20.211251 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" event={"ID":"4dc67243-5de9-4de1-bc50-df52da5facf8","Type":"ContainerStarted","Data":"68eaa4734b4824fff834cec71f6b872f04ff1760b05567a044609641eac3e4d0"} Sep 15 12:08:21.217046 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:21.217015 2582 generic.go:356] "Generic (PLEG): container finished" podID="4dc67243-5de9-4de1-bc50-df52da5facf8" containerID="68eaa4734b4824fff834cec71f6b872f04ff1760b05567a044609641eac3e4d0" exitCode=0 Sep 15 12:08:21.217453 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:21.217103 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" event={"ID":"4dc67243-5de9-4de1-bc50-df52da5facf8","Type":"ContainerDied","Data":"68eaa4734b4824fff834cec71f6b872f04ff1760b05567a044609641eac3e4d0"} Sep 15 12:08:22.225359 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:22.225318 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" event={"ID":"4dc67243-5de9-4de1-bc50-df52da5facf8","Type":"ContainerStarted","Data":"a038fb407533f0e3690dd81243fb4f6de28e9c9acdd97ee828bd93bb6901630a"} Sep 15 12:08:22.225807 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:22.225447 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:22.247807 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:22.247757 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" podStartSLOduration=1.83732871 podStartE2EDuration="6.247739379s" totalImagesPullingTime="3.358723268s" totalInitContainerRuntime="1.051687401s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:08:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:08:22.245887565 +0000 UTC m=+354.002992637" watchObservedRunningTime="2026-09-15 12:08:22.247739379 +0000 UTC m=+354.004844449" Sep 15 12:08:27.260673 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.260638 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7498d49ccd-7656n"] Sep 15 12:08:27.270201 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.270172 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.273810 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.273781 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7498d49ccd-7656n"] Sep 15 12:08:27.344910 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.344875 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-console-serving-cert\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.344910 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.344909 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-service-ca\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.345134 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.344937 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg49c\" (UniqueName: \"kubernetes.io/projected/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-kube-api-access-rg49c\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.345134 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.345014 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-console-config\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.345134 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.345097 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-console-oauth-config\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.345134 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.345122 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-oauth-serving-cert\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.345285 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.345150 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-trusted-ca-bundle\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.445688 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.445650 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-console-oauth-config\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.445688 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.445684 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-oauth-serving-cert\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.445928 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.445703 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-trusted-ca-bundle\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.445928 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.445726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-console-serving-cert\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.445928 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.445743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-service-ca\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.445928 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.445884 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg49c\" (UniqueName: \"kubernetes.io/projected/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-kube-api-access-rg49c\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.446126 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.445937 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-console-config\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.446532 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.446502 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-service-ca\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.446660 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.446582 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-oauth-serving-cert\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.446660 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.446582 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-console-config\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.447147 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.447129 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-trusted-ca-bundle\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.448377 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.448352 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-console-serving-cert\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.448377 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.448364 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-console-oauth-config\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.455416 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.455395 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg49c\" (UniqueName: \"kubernetes.io/projected/a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e-kube-api-access-rg49c\") pod \"console-7498d49ccd-7656n\" (UID: \"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e\") " pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.580935 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.580844 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:27.855548 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:27.855383 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7498d49ccd-7656n"] Sep 15 12:08:27.858184 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:08:27.858154 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2694ca0_9aeb_4e74_9f82_abd1d9f4bb0e.slice/crio-10ccc6f79d08c0ff8b71cefd022e76626a4bc5cfc7b4eeffc665c160d4600114 WatchSource:0}: Error finding container 10ccc6f79d08c0ff8b71cefd022e76626a4bc5cfc7b4eeffc665c160d4600114: Status 404 returned error can't find the container with id 10ccc6f79d08c0ff8b71cefd022e76626a4bc5cfc7b4eeffc665c160d4600114 Sep 15 12:08:28.252998 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:28.252959 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7498d49ccd-7656n" event={"ID":"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e","Type":"ContainerStarted","Data":"64e0f77675a10f6561ec523c827c093c7f349782b7f14c94520a4e11dbad5fd5"} Sep 15 12:08:28.253187 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:28.253019 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7498d49ccd-7656n" event={"ID":"a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e","Type":"ContainerStarted","Data":"10ccc6f79d08c0ff8b71cefd022e76626a4bc5cfc7b4eeffc665c160d4600114"} Sep 15 12:08:28.277902 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:28.277851 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7498d49ccd-7656n" podStartSLOduration=1.2778348849999999 podStartE2EDuration="1.277834885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:08:28.276220283 +0000 UTC m=+360.033325377" watchObservedRunningTime="2026-09-15 12:08:28.277834885 +0000 UTC m=+360.034939982" Sep 15 12:08:33.232812 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:33.232777 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b747b9585-74lgg" Sep 15 12:08:35.989986 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:35.989949 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv"] Sep 15 12:08:35.995617 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:35.995592 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:35.999621 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:35.999550 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 15 12:08:35.999621 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:35.999566 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4snpp\"" Sep 15 12:08:35.999799 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:35.999629 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 15 12:08:35.999799 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:35.999632 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 15 12:08:36.003243 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.003218 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv"] Sep 15 12:08:36.019034 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.019012 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vglzx\" (UniqueName: \"kubernetes.io/projected/6bf8bea2-da4b-468e-85a9-95c392401a04-kube-api-access-vglzx\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.019155 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.019049 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bf8bea2-da4b-468e-85a9-95c392401a04-cert\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.019155 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.019082 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6bf8bea2-da4b-468e-85a9-95c392401a04-metrics-cert\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.019155 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.019133 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6bf8bea2-da4b-468e-85a9-95c392401a04-manager-config\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.119784 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.119746 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6bf8bea2-da4b-468e-85a9-95c392401a04-metrics-cert\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.119784 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.119784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6bf8bea2-da4b-468e-85a9-95c392401a04-manager-config\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.120006 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.119846 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vglzx\" (UniqueName: \"kubernetes.io/projected/6bf8bea2-da4b-468e-85a9-95c392401a04-kube-api-access-vglzx\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.120006 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.119866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bf8bea2-da4b-468e-85a9-95c392401a04-cert\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.120596 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.120548 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6bf8bea2-da4b-468e-85a9-95c392401a04-manager-config\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.122406 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.122384 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6bf8bea2-da4b-468e-85a9-95c392401a04-metrics-cert\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.122597 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.122577 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bf8bea2-da4b-468e-85a9-95c392401a04-cert\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.135162 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.135136 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vglzx\" (UniqueName: \"kubernetes.io/projected/6bf8bea2-da4b-468e-85a9-95c392401a04-kube-api-access-vglzx\") pod \"lws-controller-manager-86c794cf5f-vjdqv\" (UID: \"6bf8bea2-da4b-468e-85a9-95c392401a04\") " pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.305909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.305822 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:36.588303 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:36.588277 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv"] Sep 15 12:08:36.590939 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:08:36.590909 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf8bea2_da4b_468e_85a9_95c392401a04.slice/crio-76f6e8f02b90728c639747216731ad55f8cfb7bce62f838d934a7037840870d5 WatchSource:0}: Error finding container 76f6e8f02b90728c639747216731ad55f8cfb7bce62f838d934a7037840870d5: Status 404 returned error can't find the container with id 76f6e8f02b90728c639747216731ad55f8cfb7bce62f838d934a7037840870d5 Sep 15 12:08:37.293260 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:37.293226 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" event={"ID":"6bf8bea2-da4b-468e-85a9-95c392401a04","Type":"ContainerStarted","Data":"76f6e8f02b90728c639747216731ad55f8cfb7bce62f838d934a7037840870d5"} Sep 15 12:08:37.581604 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:37.581522 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:37.581867 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:37.581840 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:37.588723 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:37.588699 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:38.299505 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:38.299444 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" event={"ID":"6bf8bea2-da4b-468e-85a9-95c392401a04","Type":"ContainerStarted","Data":"6fec16bda847e1d171bb607d4b473cb2d72bf584cb63575753e812c15621fb09"} Sep 15 12:08:38.299972 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:38.299660 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:38.303428 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:38.303404 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7498d49ccd-7656n" Sep 15 12:08:38.331243 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:38.331196 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" podStartSLOduration=1.817542756 podStartE2EDuration="3.331181294s" totalImagesPullingTime="1.513638538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:08:38.329590771 +0000 UTC m=+370.086695842" watchObservedRunningTime="2026-09-15 12:08:38.331181294 +0000 UTC m=+370.088286365" Sep 15 12:08:38.423631 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:38.423545 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d5d9dc4fd-dtk5s"] Sep 15 12:08:44.590447 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.590410 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc"] Sep 15 12:08:44.596374 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.596340 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.600532 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.600500 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 15 12:08:44.600677 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.600558 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 15 12:08:44.600677 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.600610 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7747l\"" Sep 15 12:08:44.600979 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.600964 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 15 12:08:44.607987 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.607960 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc"] Sep 15 12:08:44.667305 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.667264 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv"] Sep 15 12:08:44.670167 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.670140 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.681043 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.681014 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv"] Sep 15 12:08:44.700056 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700025 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.700213 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700064 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.700213 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700095 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5906647c-59ec-4391-8389-8dcc624b1b1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.700213 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700149 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.700213 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700193 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.700389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700213 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.700389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700239 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.700389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700255 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7swgr\" (UniqueName: \"kubernetes.io/projected/5906647c-59ec-4391-8389-8dcc624b1b1f-kube-api-access-7swgr\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.700389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700289 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.700389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700310 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.700389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700325 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.700389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700344 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.700389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700359 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.700389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700376 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.700723 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700397 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.700723 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700437 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.700723 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700465 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.700723 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.700481 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmfmw\" (UniqueName: \"kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-kube-api-access-nmfmw\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801237 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5906647c-59ec-4391-8389-8dcc624b1b1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.801237 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801240 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801523 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801280 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.801523 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801302 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801523 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801331 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.801523 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801357 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7swgr\" (UniqueName: \"kubernetes.io/projected/5906647c-59ec-4391-8389-8dcc624b1b1f-kube-api-access-7swgr\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.801523 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801403 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.801523 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.801523 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801523 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801516 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801542 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801568 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801602 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801651 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801700 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmfmw\" (UniqueName: \"kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-kube-api-access-nmfmw\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801726 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801773 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801804 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801856 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.801924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801897 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.802517 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.801991 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.802517 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.802035 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5906647c-59ec-4391-8389-8dcc624b1b1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.802517 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.802227 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.802517 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.802296 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.802517 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.802384 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.802517 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.802388 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.802837 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.802816 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.804339 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.804316 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.804433 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.804322 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.804946 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.804927 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.805035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.805007 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.813686 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.813659 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5906647c-59ec-4391-8389-8dcc624b1b1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.814622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.814602 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.814622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.814613 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7swgr\" (UniqueName: \"kubernetes.io/projected/5906647c-59ec-4391-8389-8dcc624b1b1f-kube-api-access-7swgr\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv\" (UID: \"5906647c-59ec-4391-8389-8dcc624b1b1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:44.814779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.814688 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmfmw\" (UniqueName: \"kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-kube-api-access-nmfmw\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.908251 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.908166 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:44.981165 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:44.981133 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:45.182751 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:45.182675 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc"] Sep 15 12:08:45.185551 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:08:45.185526 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb5b62f_46ac_4d1b_b325_c1a7b12c5596.slice/crio-a1503307831db6bbd1b58614e4c31c8b6d8e6edd0b93b14cdca9bd31d8e76c0e WatchSource:0}: Error finding container a1503307831db6bbd1b58614e4c31c8b6d8e6edd0b93b14cdca9bd31d8e76c0e: Status 404 returned error can't find the container with id a1503307831db6bbd1b58614e4c31c8b6d8e6edd0b93b14cdca9bd31d8e76c0e Sep 15 12:08:45.254741 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:45.254714 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv"] Sep 15 12:08:45.256345 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:08:45.256319 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5906647c_59ec_4391_8389_8dcc624b1b1f.slice/crio-9a64d3e7883b14f392aa1dfa96259db8e3c64fea601cae4fcbd58baf8ccb677e WatchSource:0}: Error finding container 9a64d3e7883b14f392aa1dfa96259db8e3c64fea601cae4fcbd58baf8ccb677e: Status 404 returned error can't find the container with id 9a64d3e7883b14f392aa1dfa96259db8e3c64fea601cae4fcbd58baf8ccb677e Sep 15 12:08:45.330932 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:45.330893 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" event={"ID":"5906647c-59ec-4391-8389-8dcc624b1b1f","Type":"ContainerStarted","Data":"9a64d3e7883b14f392aa1dfa96259db8e3c64fea601cae4fcbd58baf8ccb677e"} Sep 15 12:08:45.332037 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:45.332014 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" event={"ID":"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596","Type":"ContainerStarted","Data":"a1503307831db6bbd1b58614e4c31c8b6d8e6edd0b93b14cdca9bd31d8e76c0e"} Sep 15 12:08:48.104882 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.104847 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-422xq"] Sep 15 12:08:48.119597 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.119546 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:08:48.120511 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.120465 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-422xq"] Sep 15 12:08:48.124072 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.124042 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-pbrwb\"" Sep 15 12:08:48.124239 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.124049 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 15 12:08:48.157422 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.157382 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:08:48.157525 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.157457 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:08:48.157525 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.157514 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:08:48.162040 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.162011 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:08:48.162147 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.162087 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:08:48.162147 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.162130 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:08:48.238199 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.238169 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xllmb\" (UniqueName: \"kubernetes.io/projected/87db7543-c2aa-4a47-9363-9fdd047dafa4-kube-api-access-xllmb\") pod \"odh-model-controller-7998c67dc4-422xq\" (UID: \"87db7543-c2aa-4a47-9363-9fdd047dafa4\") " pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:08:48.238289 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.238227 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87db7543-c2aa-4a47-9363-9fdd047dafa4-cert\") pod \"odh-model-controller-7998c67dc4-422xq\" (UID: \"87db7543-c2aa-4a47-9363-9fdd047dafa4\") " pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:08:48.340504 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.339555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xllmb\" (UniqueName: \"kubernetes.io/projected/87db7543-c2aa-4a47-9363-9fdd047dafa4-kube-api-access-xllmb\") pod \"odh-model-controller-7998c67dc4-422xq\" (UID: \"87db7543-c2aa-4a47-9363-9fdd047dafa4\") " pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:08:48.340504 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.339632 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87db7543-c2aa-4a47-9363-9fdd047dafa4-cert\") pod \"odh-model-controller-7998c67dc4-422xq\" (UID: \"87db7543-c2aa-4a47-9363-9fdd047dafa4\") " pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:08:48.340504 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:08:48.339870 2582 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 15 12:08:48.340504 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:08:48.339952 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87db7543-c2aa-4a47-9363-9fdd047dafa4-cert podName:87db7543-c2aa-4a47-9363-9fdd047dafa4 nodeName:}" failed. No retries permitted until 2026-09-15 12:08:48.839929008 +0000 UTC m=+380.597034057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87db7543-c2aa-4a47-9363-9fdd047dafa4-cert") pod "odh-model-controller-7998c67dc4-422xq" (UID: "87db7543-c2aa-4a47-9363-9fdd047dafa4") : secret "odh-model-controller-webhook-cert" not found Sep 15 12:08:48.362542 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.356779 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" event={"ID":"5906647c-59ec-4391-8389-8dcc624b1b1f","Type":"ContainerStarted","Data":"51293a848db858466fbfed8cbee6fb5e7b73af3272caa5d0b776854eadb65d23"} Sep 15 12:08:48.362542 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.358325 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xllmb\" (UniqueName: \"kubernetes.io/projected/87db7543-c2aa-4a47-9363-9fdd047dafa4-kube-api-access-xllmb\") pod \"odh-model-controller-7998c67dc4-422xq\" (UID: \"87db7543-c2aa-4a47-9363-9fdd047dafa4\") " pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:08:48.362542 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.360290 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" event={"ID":"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596","Type":"ContainerStarted","Data":"ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a"} Sep 15 12:08:48.384104 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.384041 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" podStartSLOduration=1.485308288 podStartE2EDuration="4.384018308s" totalImagesPullingTime="2.89871002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:08:48.382919198 +0000 UTC m=+380.140024271" watchObservedRunningTime="2026-09-15 12:08:48.384018308 +0000 UTC m=+380.141123383" Sep 15 12:08:48.405088 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.405022 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" podStartSLOduration=1.430766485 podStartE2EDuration="4.404986801s" totalImagesPullingTime="2.974220316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:08:48.403848231 +0000 UTC m=+380.160953304" watchObservedRunningTime="2026-09-15 12:08:48.404986801 +0000 UTC m=+380.162091870" Sep 15 12:08:48.845092 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.845044 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87db7543-c2aa-4a47-9363-9fdd047dafa4-cert\") pod \"odh-model-controller-7998c67dc4-422xq\" (UID: \"87db7543-c2aa-4a47-9363-9fdd047dafa4\") " pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:08:48.847759 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.847720 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87db7543-c2aa-4a47-9363-9fdd047dafa4-cert\") pod \"odh-model-controller-7998c67dc4-422xq\" (UID: \"87db7543-c2aa-4a47-9363-9fdd047dafa4\") " pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:08:48.881720 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.881682 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jqns7"] Sep 15 12:08:48.884367 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.884349 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" Sep 15 12:08:48.886969 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.886943 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 15 12:08:48.887112 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.887093 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-bq5nn\"" Sep 15 12:08:48.895891 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.895870 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jqns7"] Sep 15 12:08:48.908698 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.908656 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:48.946213 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.946175 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26665ca0-d0ee-48aa-958f-add728583758-tls-certs\") pod \"model-serving-api-54d89bd5fc-jqns7\" (UID: \"26665ca0-d0ee-48aa-958f-add728583758\") " pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" Sep 15 12:08:48.946213 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.946213 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp5rg\" (UniqueName: \"kubernetes.io/projected/26665ca0-d0ee-48aa-958f-add728583758-kube-api-access-wp5rg\") pod \"model-serving-api-54d89bd5fc-jqns7\" (UID: \"26665ca0-d0ee-48aa-958f-add728583758\") " pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" Sep 15 12:08:48.982107 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:48.982053 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:49.034033 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.034000 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:08:49.047709 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.047666 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26665ca0-d0ee-48aa-958f-add728583758-tls-certs\") pod \"model-serving-api-54d89bd5fc-jqns7\" (UID: \"26665ca0-d0ee-48aa-958f-add728583758\") " pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" Sep 15 12:08:49.047837 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.047718 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp5rg\" (UniqueName: \"kubernetes.io/projected/26665ca0-d0ee-48aa-958f-add728583758-kube-api-access-wp5rg\") pod \"model-serving-api-54d89bd5fc-jqns7\" (UID: \"26665ca0-d0ee-48aa-958f-add728583758\") " pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" Sep 15 12:08:49.051083 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.051057 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26665ca0-d0ee-48aa-958f-add728583758-tls-certs\") pod \"model-serving-api-54d89bd5fc-jqns7\" (UID: \"26665ca0-d0ee-48aa-958f-add728583758\") " pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" Sep 15 12:08:49.056571 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.056547 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp5rg\" (UniqueName: \"kubernetes.io/projected/26665ca0-d0ee-48aa-958f-add728583758-kube-api-access-wp5rg\") pod \"model-serving-api-54d89bd5fc-jqns7\" (UID: \"26665ca0-d0ee-48aa-958f-add728583758\") " pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" Sep 15 12:08:49.195448 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.195364 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" Sep 15 12:08:49.306306 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.306279 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-86c794cf5f-vjdqv" Sep 15 12:08:49.319401 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.319374 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-422xq"] Sep 15 12:08:49.322101 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:08:49.322072 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87db7543_c2aa_4a47_9363_9fdd047dafa4.slice/crio-e7f268d491566d6a6171ed7a800db69de8f73a51430487f99d2453b4d5824a42 WatchSource:0}: Error finding container e7f268d491566d6a6171ed7a800db69de8f73a51430487f99d2453b4d5824a42: Status 404 returned error can't find the container with id e7f268d491566d6a6171ed7a800db69de8f73a51430487f99d2453b4d5824a42 Sep 15 12:08:49.366139 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.366101 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-422xq" event={"ID":"87db7543-c2aa-4a47-9363-9fdd047dafa4","Type":"ContainerStarted","Data":"e7f268d491566d6a6171ed7a800db69de8f73a51430487f99d2453b4d5824a42"} Sep 15 12:08:49.472752 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.472720 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jqns7"] Sep 15 12:08:49.475503 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:08:49.475456 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26665ca0_d0ee_48aa_958f_add728583758.slice/crio-b2a8a9c256deeca718e18c9ae55c30c2981a1dc39170ca5310ffee70948cd626 WatchSource:0}: Error finding container b2a8a9c256deeca718e18c9ae55c30c2981a1dc39170ca5310ffee70948cd626: Status 404 returned error can't find the container with id b2a8a9c256deeca718e18c9ae55c30c2981a1dc39170ca5310ffee70948cd626 Sep 15 12:08:49.910187 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.910105 2582 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Sep 15 12:08:49.910187 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.910160 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" podUID="cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Sep 15 12:08:49.986241 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:49.986207 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:50.374473 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:50.374391 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" event={"ID":"26665ca0-d0ee-48aa-958f-add728583758","Type":"ContainerStarted","Data":"b2a8a9c256deeca718e18c9ae55c30c2981a1dc39170ca5310ffee70948cd626"} Sep 15 12:08:50.375044 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:50.374695 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:50.375949 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:50.375922 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv" Sep 15 12:08:50.427893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:50.427850 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc"] Sep 15 12:08:50.909354 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:50.908794 2582 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Sep 15 12:08:50.909354 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:50.908982 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" podUID="cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Sep 15 12:08:51.384280 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:51.384240 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" podUID="cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" containerName="istio-proxy" containerID="cri-o://ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a" gracePeriod=30 Sep 15 12:08:54.400581 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:54.400540 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" event={"ID":"26665ca0-d0ee-48aa-958f-add728583758","Type":"ContainerStarted","Data":"c26eeb68c4e1d5ebec45d6c9753ca3943e5e6eb0739d658b6033968514576fd3"} Sep 15 12:08:54.401079 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:54.400607 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" Sep 15 12:08:54.402191 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:54.402157 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-422xq" event={"ID":"87db7543-c2aa-4a47-9363-9fdd047dafa4","Type":"ContainerStarted","Data":"a167abb00feb5cab694e275bf3f4a00e47cb43a9e552bc46496e74dba3333906"} Sep 15 12:08:54.402315 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:54.402302 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:08:54.434647 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:54.434593 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" podStartSLOduration=2.082533701 podStartE2EDuration="6.434575917s" totalImagesPullingTime="4.352042216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:08:54.432378945 +0000 UTC m=+386.189484016" watchObservedRunningTime="2026-09-15 12:08:54.434575917 +0000 UTC m=+386.191681013" Sep 15 12:08:54.473535 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:54.473454 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-422xq" podStartSLOduration=1.923892645 podStartE2EDuration="6.473437211s" totalImagesPullingTime="4.549544566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:08:54.472830137 +0000 UTC m=+386.229935211" watchObservedRunningTime="2026-09-15 12:08:54.473437211 +0000 UTC m=+386.230542282" Sep 15 12:08:55.986940 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:55.986901 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl"] Sep 15 12:08:55.990962 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:55.990943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:55.993769 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:55.993748 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 15 12:08:55.993769 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:55.993759 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 15 12:08:55.994148 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:55.994126 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 15 12:08:55.994265 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:55.994184 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zp672\"" Sep 15 12:08:56.008868 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.008839 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl"] Sep 15 12:08:56.121267 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.121231 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/091727bd-6619-4ee6-8be2-718eec0832ec-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.121267 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.121278 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/091727bd-6619-4ee6-8be2-718eec0832ec-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.121539 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.121300 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx6k6\" (UniqueName: \"kubernetes.io/projected/091727bd-6619-4ee6-8be2-718eec0832ec-kube-api-access-kx6k6\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.121539 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.121394 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/091727bd-6619-4ee6-8be2-718eec0832ec-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.121539 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.121458 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/091727bd-6619-4ee6-8be2-718eec0832ec-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.121665 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.121542 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/091727bd-6619-4ee6-8be2-718eec0832ec-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.121665 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.121569 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/091727bd-6619-4ee6-8be2-718eec0832ec-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.222627 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.222583 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/091727bd-6619-4ee6-8be2-718eec0832ec-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.222627 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.222633 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/091727bd-6619-4ee6-8be2-718eec0832ec-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.222886 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.222650 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/091727bd-6619-4ee6-8be2-718eec0832ec-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.222886 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.222783 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/091727bd-6619-4ee6-8be2-718eec0832ec-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.222886 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.222827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/091727bd-6619-4ee6-8be2-718eec0832ec-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.222886 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.222854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx6k6\" (UniqueName: \"kubernetes.io/projected/091727bd-6619-4ee6-8be2-718eec0832ec-kube-api-access-kx6k6\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.223090 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.222886 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/091727bd-6619-4ee6-8be2-718eec0832ec-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.223527 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.223412 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/091727bd-6619-4ee6-8be2-718eec0832ec-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.225143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.225114 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/091727bd-6619-4ee6-8be2-718eec0832ec-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.225335 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.225317 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/091727bd-6619-4ee6-8be2-718eec0832ec-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.225437 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.225416 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/091727bd-6619-4ee6-8be2-718eec0832ec-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.225437 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.225429 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/091727bd-6619-4ee6-8be2-718eec0832ec-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.240522 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.240439 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx6k6\" (UniqueName: \"kubernetes.io/projected/091727bd-6619-4ee6-8be2-718eec0832ec-kube-api-access-kx6k6\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.240683 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.240522 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/091727bd-6619-4ee6-8be2-718eec0832ec-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ccvgl\" (UID: \"091727bd-6619-4ee6-8be2-718eec0832ec\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.301564 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.301529 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:56.599307 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.599063 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl"] Sep 15 12:08:56.601708 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:08:56.601681 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091727bd_6619_4ee6_8be2_718eec0832ec.slice/crio-d0845c646e1f6bcf55b96167a7de12baadf1d8f2c731fb4bee79fde72d5b7d89 WatchSource:0}: Error finding container d0845c646e1f6bcf55b96167a7de12baadf1d8f2c731fb4bee79fde72d5b7d89: Status 404 returned error can't find the container with id d0845c646e1f6bcf55b96167a7de12baadf1d8f2c731fb4bee79fde72d5b7d89 Sep 15 12:08:56.643638 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.643611 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:56.727747 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.727716 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-certs\") pod \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " Sep 15 12:08:56.727747 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.727762 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-envoy\") pod \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " Sep 15 12:08:56.728001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.727783 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-socket\") pod \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " Sep 15 12:08:56.728001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.727888 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-data\") pod \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " Sep 15 12:08:56.728001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.727919 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-credential-socket\") pod \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " Sep 15 12:08:56.728001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.727945 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmfmw\" (UniqueName: \"kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-kube-api-access-nmfmw\") pod \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " Sep 15 12:08:56.728001 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.727979 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-token\") pod \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " Sep 15 12:08:56.728244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728025 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-podinfo\") pod \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " Sep 15 12:08:56.728244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728091 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" (UID: "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:08:56.728244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728146 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" (UID: "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:08:56.728244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728182 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istiod-ca-cert\") pod \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\" (UID: \"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596\") " Sep 15 12:08:56.728244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728235 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-data" (OuterVolumeSpecName: "istio-data") pod "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" (UID: "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:08:56.728469 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728353 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" (UID: "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:08:56.728618 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728559 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-credential-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:08:56.728618 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728582 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-certs\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:08:56.728618 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728598 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-workload-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:08:56.728618 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728610 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-data\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:08:56.728779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.728626 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" (UID: "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:08:56.730448 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.730427 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-kube-api-access-nmfmw" (OuterVolumeSpecName: "kube-api-access-nmfmw") pod "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" (UID: "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596"). InnerVolumeSpecName "kube-api-access-nmfmw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:08:56.730532 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.730451 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" (UID: "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 12:08:56.730600 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.730582 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" (UID: "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:08:56.730600 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.730587 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-token" (OuterVolumeSpecName: "istio-token") pod "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" (UID: "cbb5b62f-46ac-4d1b-b325-c1a7b12c5596"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:08:56.829415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.829319 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmfmw\" (UniqueName: \"kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-kube-api-access-nmfmw\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:08:56.829415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.829352 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-token\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:08:56.829415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.829362 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-podinfo\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:08:56.829415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.829370 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istiod-ca-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:08:56.829415 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:56.829379 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596-istio-envoy\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:08:57.424905 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:57.424870 2582 generic.go:356] "Generic (PLEG): container finished" podID="cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" containerID="ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a" exitCode=0 Sep 15 12:08:57.425380 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:57.424919 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" Sep 15 12:08:57.425380 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:57.424961 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" event={"ID":"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596","Type":"ContainerDied","Data":"ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a"} Sep 15 12:08:57.425380 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:57.425032 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc" event={"ID":"cbb5b62f-46ac-4d1b-b325-c1a7b12c5596","Type":"ContainerDied","Data":"a1503307831db6bbd1b58614e4c31c8b6d8e6edd0b93b14cdca9bd31d8e76c0e"} Sep 15 12:08:57.425380 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:57.425066 2582 scope.go:117] "RemoveContainer" containerID="ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a" Sep 15 12:08:57.426631 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:57.426603 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" event={"ID":"091727bd-6619-4ee6-8be2-718eec0832ec","Type":"ContainerStarted","Data":"d0845c646e1f6bcf55b96167a7de12baadf1d8f2c731fb4bee79fde72d5b7d89"} Sep 15 12:08:57.435634 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:57.435615 2582 scope.go:117] "RemoveContainer" containerID="ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a" Sep 15 12:08:57.436105 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:08:57.436077 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a\": container with ID starting with ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a not found: ID does not exist" containerID="ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a" Sep 15 12:08:57.436216 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:57.436115 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a"} err="failed to get container status \"ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a\": rpc error: code = NotFound desc = could not find container \"ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a\": container with ID starting with ac3ac5bac124e33f2191b6efa00ee297d9d19bed166434b551fc9c2194fd8c7a not found: ID does not exist" Sep 15 12:08:57.463891 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:57.463851 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc"] Sep 15 12:08:57.474826 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:57.474793 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-cl7pc"] Sep 15 12:08:58.712871 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:58.712837 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" path="/var/lib/kubelet/pods/cbb5b62f-46ac-4d1b-b325-c1a7b12c5596/volumes" Sep 15 12:08:58.957967 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:58.957913 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:08:58.958078 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:58.958015 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:08:59.441992 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:59.441902 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" event={"ID":"091727bd-6619-4ee6-8be2-718eec0832ec","Type":"ContainerStarted","Data":"9ee8909e4d4d7bc80af976a440966004e8406948a3528fa64eb582fef8fafcb0"} Sep 15 12:08:59.442162 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:59.442033 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:08:59.465431 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:08:59.465385 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" podStartSLOduration=2.1120752720000002 podStartE2EDuration="4.46536876s" totalImagesPullingTime="2.353293488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:08:59.465334005 +0000 UTC m=+391.222439076" watchObservedRunningTime="2026-09-15 12:08:59.46536876 +0000 UTC m=+391.222473828" Sep 15 12:09:00.453844 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:00.453809 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccvgl" Sep 15 12:09:03.443246 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.443188 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d5d9dc4fd-dtk5s" podUID="5d0e40fe-9af9-49e0-babd-8f6b67bfe050" containerName="console" containerID="cri-o://b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc" gracePeriod=15 Sep 15 12:09:03.697025 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.696945 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:09:03.791811 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.791778 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-config\") pod \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " Sep 15 12:09:03.791992 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.791832 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-oauth-config\") pod \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " Sep 15 12:09:03.791992 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.791854 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdk82\" (UniqueName: \"kubernetes.io/projected/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-kube-api-access-xdk82\") pod \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " Sep 15 12:09:03.791992 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.791877 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-trusted-ca-bundle\") pod \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " Sep 15 12:09:03.791992 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.791916 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-service-ca\") pod \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " Sep 15 12:09:03.791992 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.791950 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-serving-cert\") pod \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " Sep 15 12:09:03.792244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.792011 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-oauth-serving-cert\") pod \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\" (UID: \"5d0e40fe-9af9-49e0-babd-8f6b67bfe050\") " Sep 15 12:09:03.792369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.792329 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-config" (OuterVolumeSpecName: "console-config") pod "5d0e40fe-9af9-49e0-babd-8f6b67bfe050" (UID: "5d0e40fe-9af9-49e0-babd-8f6b67bfe050"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:09:03.792453 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.792417 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5d0e40fe-9af9-49e0-babd-8f6b67bfe050" (UID: "5d0e40fe-9af9-49e0-babd-8f6b67bfe050"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:09:03.792453 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.792434 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-service-ca" (OuterVolumeSpecName: "service-ca") pod "5d0e40fe-9af9-49e0-babd-8f6b67bfe050" (UID: "5d0e40fe-9af9-49e0-babd-8f6b67bfe050"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:09:03.792453 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.792442 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5d0e40fe-9af9-49e0-babd-8f6b67bfe050" (UID: "5d0e40fe-9af9-49e0-babd-8f6b67bfe050"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:09:03.794178 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.794151 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5d0e40fe-9af9-49e0-babd-8f6b67bfe050" (UID: "5d0e40fe-9af9-49e0-babd-8f6b67bfe050"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:09:03.794425 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.794404 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5d0e40fe-9af9-49e0-babd-8f6b67bfe050" (UID: "5d0e40fe-9af9-49e0-babd-8f6b67bfe050"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:09:03.794651 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.794632 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-kube-api-access-xdk82" (OuterVolumeSpecName: "kube-api-access-xdk82") pod "5d0e40fe-9af9-49e0-babd-8f6b67bfe050" (UID: "5d0e40fe-9af9-49e0-babd-8f6b67bfe050"). InnerVolumeSpecName "kube-api-access-xdk82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:09:03.893447 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.893404 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-oauth-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:09:03.893447 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.893439 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdk82\" (UniqueName: \"kubernetes.io/projected/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-kube-api-access-xdk82\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:09:03.893447 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.893449 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-trusted-ca-bundle\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:09:03.893447 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.893458 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:09:03.893757 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.893467 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-serving-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:09:03.893757 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.893478 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-oauth-serving-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:09:03.893757 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:03.893513 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d0e40fe-9af9-49e0-babd-8f6b67bfe050-console-config\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:09:04.470198 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:04.470167 2582 generic.go:356] "Generic (PLEG): container finished" podID="5d0e40fe-9af9-49e0-babd-8f6b67bfe050" containerID="b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc" exitCode=0 Sep 15 12:09:04.470628 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:04.470213 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5d9dc4fd-dtk5s" Sep 15 12:09:04.470628 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:04.470251 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5d9dc4fd-dtk5s" event={"ID":"5d0e40fe-9af9-49e0-babd-8f6b67bfe050","Type":"ContainerDied","Data":"b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc"} Sep 15 12:09:04.470628 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:04.470301 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5d9dc4fd-dtk5s" event={"ID":"5d0e40fe-9af9-49e0-babd-8f6b67bfe050","Type":"ContainerDied","Data":"16983096a6c4493abf3465c5d2c5879353036bfdbf8d95ffaf652132d72958ff"} Sep 15 12:09:04.470628 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:04.470324 2582 scope.go:117] "RemoveContainer" containerID="b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc" Sep 15 12:09:04.479432 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:04.479416 2582 scope.go:117] "RemoveContainer" containerID="b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc" Sep 15 12:09:04.479720 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:09:04.479703 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc\": container with ID starting with b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc not found: ID does not exist" containerID="b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc" Sep 15 12:09:04.479770 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:04.479729 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc"} err="failed to get container status \"b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc\": rpc error: code = NotFound desc = could not find container \"b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc\": container with ID starting with b1639ae82f29fc8b8ae31e666c8847ba5af3bfc222faf79779e710e11a012ccc not found: ID does not exist" Sep 15 12:09:04.493655 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:04.493624 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d5d9dc4fd-dtk5s"] Sep 15 12:09:04.498178 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:04.498153 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d5d9dc4fd-dtk5s"] Sep 15 12:09:04.713840 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:04.713773 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d0e40fe-9af9-49e0-babd-8f6b67bfe050" path="/var/lib/kubelet/pods/5d0e40fe-9af9-49e0-babd-8f6b67bfe050/volumes" Sep 15 12:09:05.408962 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:05.408935 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-422xq" Sep 15 12:09:05.411026 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:09:05.411005 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-jqns7" Sep 15 12:11:07.716872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.716838 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv"] Sep 15 12:11:07.717347 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.717217 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d0e40fe-9af9-49e0-babd-8f6b67bfe050" containerName="console" Sep 15 12:11:07.717347 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.717230 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d0e40fe-9af9-49e0-babd-8f6b67bfe050" containerName="console" Sep 15 12:11:07.717347 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.717237 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" containerName="istio-proxy" Sep 15 12:11:07.717347 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.717242 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" containerName="istio-proxy" Sep 15 12:11:07.717347 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.717317 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbb5b62f-46ac-4d1b-b325-c1a7b12c5596" containerName="istio-proxy" Sep 15 12:11:07.717347 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.717331 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d0e40fe-9af9-49e0-babd-8f6b67bfe050" containerName="console" Sep 15 12:11:07.720274 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.720255 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.723069 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.723047 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7jp9w\"" Sep 15 12:11:07.733800 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.733772 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv"] Sep 15 12:11:07.786427 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.786397 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c61cf7af-1e83-4674-b1b5-8795276fd46f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.786628 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.786437 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.786628 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.786462 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.786628 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.786582 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.786628 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.786607 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll9bl\" (UniqueName: \"kubernetes.io/projected/c61cf7af-1e83-4674-b1b5-8795276fd46f-kube-api-access-ll9bl\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.786799 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.786636 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.786799 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.786666 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.786799 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.786700 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.786799 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.786722 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887414 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887454 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887454 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll9bl\" (UniqueName: \"kubernetes.io/projected/c61cf7af-1e83-4674-b1b5-8795276fd46f-kube-api-access-ll9bl\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887752 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887503 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887752 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887524 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887752 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887678 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887752 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887730 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887776 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c61cf7af-1e83-4674-b1b5-8795276fd46f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887841 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887879 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887909 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.887953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.887930 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.888216 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.888020 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.888216 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.888210 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.888556 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.888535 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c61cf7af-1e83-4674-b1b5-8795276fd46f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.889816 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.889794 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.890312 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.890290 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.899324 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.899301 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c61cf7af-1e83-4674-b1b5-8795276fd46f-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:07.899513 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:07.899470 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll9bl\" (UniqueName: \"kubernetes.io/projected/c61cf7af-1e83-4674-b1b5-8795276fd46f-kube-api-access-ll9bl\") pod \"maas-default-gateway-openshift-default-5ccb968c66-nv8bv\" (UID: \"c61cf7af-1e83-4674-b1b5-8795276fd46f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:08.030236 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:08.030159 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:08.307327 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:08.307272 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv"] Sep 15 12:11:08.309504 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:11:08.309461 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61cf7af_1e83_4674_b1b5_8795276fd46f.slice/crio-df947aacecd9975a9b59585083b0eee4d27912ebc9fb87dd197a0275f7e473d8 WatchSource:0}: Error finding container df947aacecd9975a9b59585083b0eee4d27912ebc9fb87dd197a0275f7e473d8: Status 404 returned error can't find the container with id df947aacecd9975a9b59585083b0eee4d27912ebc9fb87dd197a0275f7e473d8 Sep 15 12:11:08.311452 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:08.311419 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:11:08.311565 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:08.311529 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:11:08.311634 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:08.311585 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:11:09.036149 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:09.036119 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" event={"ID":"c61cf7af-1e83-4674-b1b5-8795276fd46f","Type":"ContainerStarted","Data":"b3b77024e6fafd9b7ef55a21fc04a119a572599f59b5e2f1ce43f959ec966b04"} Sep 15 12:11:09.036568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:09.036169 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" event={"ID":"c61cf7af-1e83-4674-b1b5-8795276fd46f","Type":"ContainerStarted","Data":"df947aacecd9975a9b59585083b0eee4d27912ebc9fb87dd197a0275f7e473d8"} Sep 15 12:11:09.059700 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:09.059653 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" podStartSLOduration=2.059637845 podStartE2EDuration="2.059637845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:11:09.057727163 +0000 UTC m=+520.814832229" watchObservedRunningTime="2026-09-15 12:11:09.059637845 +0000 UTC m=+520.816742917" Sep 15 12:11:10.030347 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:10.030301 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:10.035239 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:10.035210 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:10.040287 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:10.040262 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:11:10.041024 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:11:10.041007 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:12:31.355221 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.355182 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-clfp7"] Sep 15 12:12:31.358364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.358348 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" Sep 15 12:12:31.361590 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.361572 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cphvm\"" Sep 15 12:12:31.361916 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.361898 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 15 12:12:31.373250 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.373226 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-clfp7"] Sep 15 12:12:31.438193 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.438163 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snz4j\" (UniqueName: \"kubernetes.io/projected/bd363dc1-4a24-4aed-83d0-d5bcbdedd04f-kube-api-access-snz4j\") pod \"payload-processing-5b54b7fbb7-clfp7\" (UID: \"bd363dc1-4a24-4aed-83d0-d5bcbdedd04f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" Sep 15 12:12:31.438357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.438270 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd363dc1-4a24-4aed-83d0-d5bcbdedd04f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-clfp7\" (UID: \"bd363dc1-4a24-4aed-83d0-d5bcbdedd04f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" Sep 15 12:12:31.539677 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.539627 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd363dc1-4a24-4aed-83d0-d5bcbdedd04f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-clfp7\" (UID: \"bd363dc1-4a24-4aed-83d0-d5bcbdedd04f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" Sep 15 12:12:31.539677 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.539695 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snz4j\" (UniqueName: \"kubernetes.io/projected/bd363dc1-4a24-4aed-83d0-d5bcbdedd04f-kube-api-access-snz4j\") pod \"payload-processing-5b54b7fbb7-clfp7\" (UID: \"bd363dc1-4a24-4aed-83d0-d5bcbdedd04f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" Sep 15 12:12:31.540398 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.540372 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd363dc1-4a24-4aed-83d0-d5bcbdedd04f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-clfp7\" (UID: \"bd363dc1-4a24-4aed-83d0-d5bcbdedd04f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" Sep 15 12:12:31.549157 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.549126 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snz4j\" (UniqueName: \"kubernetes.io/projected/bd363dc1-4a24-4aed-83d0-d5bcbdedd04f-kube-api-access-snz4j\") pod \"payload-processing-5b54b7fbb7-clfp7\" (UID: \"bd363dc1-4a24-4aed-83d0-d5bcbdedd04f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" Sep 15 12:12:31.667913 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.667825 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" Sep 15 12:12:31.947990 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:31.947964 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-clfp7"] Sep 15 12:12:31.950402 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:12:31.950374 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd363dc1_4a24_4aed_83d0_d5bcbdedd04f.slice/crio-4d874c16117eeaa82c8f8afb03c95ba2c7ec67e7b568d0dc9d848e1fff80a1de WatchSource:0}: Error finding container 4d874c16117eeaa82c8f8afb03c95ba2c7ec67e7b568d0dc9d848e1fff80a1de: Status 404 returned error can't find the container with id 4d874c16117eeaa82c8f8afb03c95ba2c7ec67e7b568d0dc9d848e1fff80a1de Sep 15 12:12:32.414231 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:32.414180 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" event={"ID":"bd363dc1-4a24-4aed-83d0-d5bcbdedd04f","Type":"ContainerStarted","Data":"4d874c16117eeaa82c8f8afb03c95ba2c7ec67e7b568d0dc9d848e1fff80a1de"} Sep 15 12:12:34.432706 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:34.432665 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" event={"ID":"bd363dc1-4a24-4aed-83d0-d5bcbdedd04f","Type":"ContainerStarted","Data":"86ef9cbb31a88afac87ba5cd8ac2b1723b5a795f56c85c0b976604bdb5ea3efd"} Sep 15 12:12:34.433098 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:34.432809 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" Sep 15 12:12:34.450355 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:34.450302 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" podStartSLOduration=1.551312175 podStartE2EDuration="3.450255005s" totalImagesPullingTime="1.89894283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:12:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:12:34.44954264 +0000 UTC m=+606.206647732" watchObservedRunningTime="2026-09-15 12:12:34.450255005 +0000 UTC m=+606.207360075" Sep 15 12:12:40.798845 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:40.798337 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-nv8bv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": read tcp 10.134.0.2:33244->10.134.0.30:15021: read: connection reset by peer" start-of-body= Sep 15 12:12:40.798845 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:40.798401 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" podUID="c61cf7af-1e83-4674-b1b5-8795276fd46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": read tcp 10.134.0.2:33244->10.134.0.30:15021: read: connection reset by peer" Sep 15 12:12:41.467973 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:41.467937 2582 generic.go:356] "Generic (PLEG): container finished" podID="c61cf7af-1e83-4674-b1b5-8795276fd46f" containerID="b3b77024e6fafd9b7ef55a21fc04a119a572599f59b5e2f1ce43f959ec966b04" exitCode=0 Sep 15 12:12:41.468178 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:41.468014 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" event={"ID":"c61cf7af-1e83-4674-b1b5-8795276fd46f","Type":"ContainerDied","Data":"b3b77024e6fafd9b7ef55a21fc04a119a572599f59b5e2f1ce43f959ec966b04"} Sep 15 12:12:41.468450 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:41.468433 2582 scope.go:117] "RemoveContainer" containerID="b3b77024e6fafd9b7ef55a21fc04a119a572599f59b5e2f1ce43f959ec966b04" Sep 15 12:12:41.469526 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:41.469498 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:12:41.469586 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:41.469567 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:12:41.469621 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:41.469600 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:12:42.474046 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:42.474013 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" event={"ID":"c61cf7af-1e83-4674-b1b5-8795276fd46f","Type":"ContainerStarted","Data":"52a2f6046321384e680d043cf1853d6b290e7f0930ddd87a0d547b2b981642a2"} Sep 15 12:12:43.031349 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:43.031305 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:12:43.031591 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:43.031568 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-nv8bv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Sep 15 12:12:43.031643 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:43.031616 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" podUID="c61cf7af-1e83-4674-b1b5-8795276fd46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Sep 15 12:12:44.031253 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:44.031216 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-nv8bv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Sep 15 12:12:44.031696 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:44.031272 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" podUID="c61cf7af-1e83-4674-b1b5-8795276fd46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Sep 15 12:12:45.031230 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:45.031195 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-nv8bv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Sep 15 12:12:45.031412 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:45.031256 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" podUID="c61cf7af-1e83-4674-b1b5-8795276fd46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Sep 15 12:12:45.440176 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:45.440092 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-clfp7" Sep 15 12:12:46.031312 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.031272 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-nv8bv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Sep 15 12:12:46.031724 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.031349 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" podUID="c61cf7af-1e83-4674-b1b5-8795276fd46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Sep 15 12:12:46.319570 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.315629 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w"] Sep 15 12:12:46.333526 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.333471 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w"] Sep 15 12:12:46.333776 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.333612 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.337011 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.336987 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 15 12:12:46.337193 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.337036 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 15 12:12:46.337963 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.337940 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 15 12:12:46.338187 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.338170 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-c9vjk\"" Sep 15 12:12:46.498246 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.498213 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3174195-cb86-409d-ba83-4f24db34a882-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.498423 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.498265 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.498423 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.498309 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s54g\" (UniqueName: \"kubernetes.io/projected/b3174195-cb86-409d-ba83-4f24db34a882-kube-api-access-4s54g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.498423 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.498397 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.498423 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.498422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.498600 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.498518 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.599878 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.599772 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.599878 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.599839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.599878 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.599871 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3174195-cb86-409d-ba83-4f24db34a882-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.600132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.599905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.600132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.599922 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s54g\" (UniqueName: \"kubernetes.io/projected/b3174195-cb86-409d-ba83-4f24db34a882-kube-api-access-4s54g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.600132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.599959 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.600274 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.600236 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.600339 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.600305 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.600392 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.600348 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.614511 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.608585 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3174195-cb86-409d-ba83-4f24db34a882-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.614511 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.608786 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3174195-cb86-409d-ba83-4f24db34a882-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.614511 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.614086 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s54g\" (UniqueName: \"kubernetes.io/projected/b3174195-cb86-409d-ba83-4f24db34a882-kube-api-access-4s54g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w\" (UID: \"b3174195-cb86-409d-ba83-4f24db34a882\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.645971 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.645933 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:46.933114 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:46.933085 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w"] Sep 15 12:12:46.935611 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:12:46.935581 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3174195_cb86_409d_ba83_4f24db34a882.slice/crio-2b4e9c1fe084826860906bfbc11df39be029b7718c84f4c0b02a7527e05f1a71 WatchSource:0}: Error finding container 2b4e9c1fe084826860906bfbc11df39be029b7718c84f4c0b02a7527e05f1a71: Status 404 returned error can't find the container with id 2b4e9c1fe084826860906bfbc11df39be029b7718c84f4c0b02a7527e05f1a71 Sep 15 12:12:47.030888 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:47.030849 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-nv8bv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Sep 15 12:12:47.031058 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:47.030907 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" podUID="c61cf7af-1e83-4674-b1b5-8795276fd46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Sep 15 12:12:47.501458 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:47.501416 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" event={"ID":"b3174195-cb86-409d-ba83-4f24db34a882","Type":"ContainerStarted","Data":"2b4e9c1fe084826860906bfbc11df39be029b7718c84f4c0b02a7527e05f1a71"} Sep 15 12:12:48.035202 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:48.035170 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:12:48.035809 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:48.035782 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:12:48.036678 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:48.036645 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-nv8bv" Sep 15 12:12:48.507448 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:48.507413 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" event={"ID":"b3174195-cb86-409d-ba83-4f24db34a882","Type":"ContainerStarted","Data":"83169feec2a31d9e970b6d876ad921a804348fc85aa0b7cfb8b0591fbdaf7d79"} Sep 15 12:12:48.529066 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:48.529017 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" podStartSLOduration=1.325739745 podStartE2EDuration="2.529003172s" totalImagesPullingTime="1.203263427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:12:48.52851652 +0000 UTC m=+620.285621592" watchObservedRunningTime="2026-09-15 12:12:48.529003172 +0000 UTC m=+620.286108243" Sep 15 12:12:56.646792 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:56.646750 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:56.647350 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:56.646846 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:56.659090 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:56.659062 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:12:57.560212 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:12:57.560184 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w" Sep 15 12:13:29.202338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.202305 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q"] Sep 15 12:13:29.206007 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.205986 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.208531 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.208501 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 15 12:13:29.216206 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.216181 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q"] Sep 15 12:13:29.319891 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.319853 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.319891 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.319893 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.320125 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.319910 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.320125 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.319990 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c62e7e15-1524-490f-b34b-505dfe2e45cf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.320125 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.320039 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.320125 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.320081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgjwx\" (UniqueName: \"kubernetes.io/projected/c62e7e15-1524-490f-b34b-505dfe2e45cf-kube-api-access-lgjwx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.421253 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.421214 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.421253 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.421259 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c62e7e15-1524-490f-b34b-505dfe2e45cf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.421541 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.421285 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.421541 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.421319 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjwx\" (UniqueName: \"kubernetes.io/projected/c62e7e15-1524-490f-b34b-505dfe2e45cf-kube-api-access-lgjwx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.421541 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.421392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.421721 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.421549 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.421721 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.421669 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.421833 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.421767 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.421874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.421861 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.423717 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.423685 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c62e7e15-1524-490f-b34b-505dfe2e45cf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.424024 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.424008 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c62e7e15-1524-490f-b34b-505dfe2e45cf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.431029 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.431001 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgjwx\" (UniqueName: \"kubernetes.io/projected/c62e7e15-1524-490f-b34b-505dfe2e45cf-kube-api-access-lgjwx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s8m2q\" (UID: \"c62e7e15-1524-490f-b34b-505dfe2e45cf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.518939 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.518897 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:29.791977 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.791953 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q"] Sep 15 12:13:29.793982 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:13:29.793951 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc62e7e15_1524_490f_b34b_505dfe2e45cf.slice/crio-5271439aba48bf7a8c7ad23b1cf83a96b4069ff940799fce766f1cbe12644a7c WatchSource:0}: Error finding container 5271439aba48bf7a8c7ad23b1cf83a96b4069ff940799fce766f1cbe12644a7c: Status 404 returned error can't find the container with id 5271439aba48bf7a8c7ad23b1cf83a96b4069ff940799fce766f1cbe12644a7c Sep 15 12:13:29.795416 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:29.795396 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 12:13:30.700112 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:30.700073 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" event={"ID":"c62e7e15-1524-490f-b34b-505dfe2e45cf","Type":"ContainerStarted","Data":"a2df5ff682f7da5fbe5eb8420a214e1d49b8806d7f951a26d84f0103612ca2e5"} Sep 15 12:13:30.700538 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:30.700132 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" event={"ID":"c62e7e15-1524-490f-b34b-505dfe2e45cf","Type":"ContainerStarted","Data":"5271439aba48bf7a8c7ad23b1cf83a96b4069ff940799fce766f1cbe12644a7c"} Sep 15 12:13:30.721640 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:30.721583 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" podStartSLOduration=1.5360783649999998 podStartE2EDuration="1.721564451s" totalImagesPullingTime="185.486086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:13:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:13:30.720569505 +0000 UTC m=+662.477674576" watchObservedRunningTime="2026-09-15 12:13:30.721564451 +0000 UTC m=+662.478669519" Sep 15 12:13:39.519342 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:39.519300 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:39.519845 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:39.519407 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:39.531703 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:39.531680 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:39.755155 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:39.755125 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s8m2q" Sep 15 12:13:40.511919 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.511887 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2"] Sep 15 12:13:40.515545 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.515520 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.519228 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.519204 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 15 12:13:40.526951 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.526930 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2"] Sep 15 12:13:40.612833 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.612794 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.613009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.612884 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.613009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.612924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.613009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.612968 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3a19858-3f4e-4990-93e2-b0d62bbdf757-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.613009 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.612995 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d2j6\" (UniqueName: \"kubernetes.io/projected/e3a19858-3f4e-4990-93e2-b0d62bbdf757-kube-api-access-8d2j6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.613150 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.613043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.713822 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.713786 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.714000 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.713831 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.714000 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.713915 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.714000 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.713973 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3a19858-3f4e-4990-93e2-b0d62bbdf757-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.714164 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.714000 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d2j6\" (UniqueName: \"kubernetes.io/projected/e3a19858-3f4e-4990-93e2-b0d62bbdf757-kube-api-access-8d2j6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.714164 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.714049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.714266 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.714204 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.714315 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.714299 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.714379 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.714355 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.716148 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.716132 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3a19858-3f4e-4990-93e2-b0d62bbdf757-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.716542 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.716519 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3a19858-3f4e-4990-93e2-b0d62bbdf757-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.722035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.722001 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d2j6\" (UniqueName: \"kubernetes.io/projected/e3a19858-3f4e-4990-93e2-b0d62bbdf757-kube-api-access-8d2j6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2\" (UID: \"e3a19858-3f4e-4990-93e2-b0d62bbdf757\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:40.827544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:40.827434 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:41.102827 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:41.102801 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2"] Sep 15 12:13:41.105038 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:13:41.105009 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3a19858_3f4e_4990_93e2_b0d62bbdf757.slice/crio-d84211b5f8ff093c827b816ddaf82b30e3537b21b8ca40d78720c85e0fb84a54 WatchSource:0}: Error finding container d84211b5f8ff093c827b816ddaf82b30e3537b21b8ca40d78720c85e0fb84a54: Status 404 returned error can't find the container with id d84211b5f8ff093c827b816ddaf82b30e3537b21b8ca40d78720c85e0fb84a54 Sep 15 12:13:41.755609 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:41.755571 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" event={"ID":"e3a19858-3f4e-4990-93e2-b0d62bbdf757","Type":"ContainerStarted","Data":"ba8f469a3d6ab74cce63e92b8ad99a9e67c070821b270741e0dbb938dab7a9bb"} Sep 15 12:13:41.755985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:41.755626 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" event={"ID":"e3a19858-3f4e-4990-93e2-b0d62bbdf757","Type":"ContainerStarted","Data":"d84211b5f8ff093c827b816ddaf82b30e3537b21b8ca40d78720c85e0fb84a54"} Sep 15 12:13:41.774913 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:41.774862 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" podStartSLOduration=1.5701718040000001 podStartE2EDuration="1.774845102s" totalImagesPullingTime="204.673298ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:13:41.773181357 +0000 UTC m=+673.530286429" watchObservedRunningTime="2026-09-15 12:13:41.774845102 +0000 UTC m=+673.531950173" Sep 15 12:13:50.828381 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:50.828340 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:50.828865 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:50.828404 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:50.841097 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:50.841075 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:13:51.819537 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:13:51.819473 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2" Sep 15 12:15:00.172086 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:00.172051 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf"] Sep 15 12:15:00.175615 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:00.175597 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf" Sep 15 12:15:00.178919 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:00.178891 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:15:00.184666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:00.184626 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf"] Sep 15 12:15:00.186044 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:00.186025 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf" Sep 15 12:15:00.463803 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:00.463778 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf"] Sep 15 12:15:00.466359 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:00.466321 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df38f6a_67bb_4d6c_b088_de84728e0db6.slice/crio-61dd914c4b87cdd19bb48c965a81eb0e7d63067cde314e944cfdeadb78ea3a6a WatchSource:0}: Error finding container 61dd914c4b87cdd19bb48c965a81eb0e7d63067cde314e944cfdeadb78ea3a6a: Status 404 returned error can't find the container with id 61dd914c4b87cdd19bb48c965a81eb0e7d63067cde314e944cfdeadb78ea3a6a Sep 15 12:15:01.132664 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:01.132623 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf" event={"ID":"6df38f6a-67bb-4d6c-b088-de84728e0db6","Type":"ContainerStarted","Data":"61dd914c4b87cdd19bb48c965a81eb0e7d63067cde314e944cfdeadb78ea3a6a"} Sep 15 12:15:03.145577 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:03.145539 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf" event={"ID":"6df38f6a-67bb-4d6c-b088-de84728e0db6","Type":"ContainerStarted","Data":"90bfd00e6a2733eb8b4b9b52f7068d9ff2c3104203b6051ddbcfcad62ec1314b"} Sep 15 12:15:03.164280 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:03.164181 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf" podStartSLOduration=0.978211395 podStartE2EDuration="3.164165305s" totalImagesPullingTime="2.18595391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:03.163364559 +0000 UTC m=+754.920469641" watchObservedRunningTime="2026-09-15 12:15:03.164165305 +0000 UTC m=+754.921270376" Sep 15 12:15:04.152311 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:04.152279 2582 generic.go:356] "Generic (PLEG): container finished" podID="6df38f6a-67bb-4d6c-b088-de84728e0db6" containerID="90bfd00e6a2733eb8b4b9b52f7068d9ff2c3104203b6051ddbcfcad62ec1314b" exitCode=0 Sep 15 12:15:04.152760 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:04.152362 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf" event={"ID":"6df38f6a-67bb-4d6c-b088-de84728e0db6","Type":"ContainerDied","Data":"90bfd00e6a2733eb8b4b9b52f7068d9ff2c3104203b6051ddbcfcad62ec1314b"} Sep 15 12:15:05.295823 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:05.295797 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf" Sep 15 12:15:06.169402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:06.169366 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf" Sep 15 12:15:06.169668 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:06.169387 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824575-lfcvf" event={"ID":"6df38f6a-67bb-4d6c-b088-de84728e0db6","Type":"ContainerDied","Data":"61dd914c4b87cdd19bb48c965a81eb0e7d63067cde314e944cfdeadb78ea3a6a"} Sep 15 12:15:06.169668 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:06.169523 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61dd914c4b87cdd19bb48c965a81eb0e7d63067cde314e944cfdeadb78ea3a6a" Sep 15 12:15:19.088146 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.088111 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5"] Sep 15 12:15:19.088580 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.088546 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df38f6a-67bb-4d6c-b088-de84728e0db6" containerName="cleanup" Sep 15 12:15:19.088580 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.088559 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df38f6a-67bb-4d6c-b088-de84728e0db6" containerName="cleanup" Sep 15 12:15:19.088731 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.088619 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6df38f6a-67bb-4d6c-b088-de84728e0db6" containerName="cleanup" Sep 15 12:15:19.091630 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.091607 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.094625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.094593 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4a9453-openshift-default-dockercfg-bshsl\"" Sep 15 12:15:19.104723 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.104695 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5"] Sep 15 12:15:19.222574 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.222531 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-podinfo\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.222774 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.222601 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-data\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.222774 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.222694 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istiod-ca-cert\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.222903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.222814 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-token\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.222903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.222848 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-certs\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.222903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.222880 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwg9h\" (UniqueName: \"kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-kube-api-access-lwg9h\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.223063 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.222983 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-socket\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.223063 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.223033 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-envoy\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.223158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.223063 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-credential-socket\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.323624 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.323585 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-token\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.323826 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.323641 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-certs\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.323826 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.323668 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwg9h\" (UniqueName: \"kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-kube-api-access-lwg9h\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.323826 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.323708 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-socket\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.323826 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.323724 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-envoy\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.323826 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.323740 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-credential-socket\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.323826 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.323792 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-podinfo\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.324148 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.323828 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-data\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.324148 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.323861 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istiod-ca-cert\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.324258 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.324152 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-certs\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.324335 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.324309 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-credential-socket\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.324381 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.324341 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-data\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.324516 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.324473 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-socket\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.324939 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.324913 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istiod-ca-cert\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.326281 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.326254 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-envoy\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.326808 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.326788 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-podinfo\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.333026 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.332995 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwg9h\" (UniqueName: \"kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-kube-api-access-lwg9h\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.334063 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.334040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-token\") pod \"e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.403332 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.403238 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:19.681378 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.681356 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5"] Sep 15 12:15:19.683698 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:19.683671 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59fd9969_fdeb_4a2b_988e_650fb17da2cc.slice/crio-231a8ba910a0285cd456bf777380ea953ada3bab0f688ab018bc0d9b6b70c306 WatchSource:0}: Error finding container 231a8ba910a0285cd456bf777380ea953ada3bab0f688ab018bc0d9b6b70c306: Status 404 returned error can't find the container with id 231a8ba910a0285cd456bf777380ea953ada3bab0f688ab018bc0d9b6b70c306 Sep 15 12:15:19.685759 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.685720 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:19.685854 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.685794 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:19.685854 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.685829 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:19.982357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.982323 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6"] Sep 15 12:15:19.986175 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.986152 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:19.988764 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:19.988742 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-680229-openshift-default-dockercfg-7stsl\"" Sep 15 12:15:20.000243 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.000150 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6"] Sep 15 12:15:20.031226 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.031195 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2736624f-181c-4b3b-9088-3b08bde5c003-istio-podinfo\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.031404 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.031249 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-istio-token\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.031404 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.031281 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2736624f-181c-4b3b-9088-3b08bde5c003-istiod-ca-cert\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.031404 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.031310 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-certs\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.031404 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.031340 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-socket\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.031598 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.031421 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-credential-socket\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.031598 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.031460 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9l5g\" (UniqueName: \"kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-kube-api-access-t9l5g\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.031598 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.031531 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-data\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.031598 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.031560 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-envoy\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.132201 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132164 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-data\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.132730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132220 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-envoy\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.132730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132278 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2736624f-181c-4b3b-9088-3b08bde5c003-istio-podinfo\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.132730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132350 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-istio-token\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.132730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132388 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2736624f-181c-4b3b-9088-3b08bde5c003-istiod-ca-cert\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.132730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132420 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-certs\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.132730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132459 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-socket\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.132730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-credential-socket\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.132730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132589 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9l5g\" (UniqueName: \"kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-kube-api-access-t9l5g\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.132730 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132663 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-data\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.133296 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.132873 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-socket\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.133296 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.133009 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-credential-socket\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.133406 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.133303 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2736624f-181c-4b3b-9088-3b08bde5c003-istiod-ca-cert\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.133406 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.133395 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-certs\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.135280 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.135255 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2736624f-181c-4b3b-9088-3b08bde5c003-istio-podinfo\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.135418 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.135395 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-envoy\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.141695 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.141664 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9l5g\" (UniqueName: \"kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-kube-api-access-t9l5g\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.141837 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.141817 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-istio-token\") pod \"e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.238691 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.238603 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" event={"ID":"59fd9969-fdeb-4a2b-988e-650fb17da2cc","Type":"ContainerStarted","Data":"2acf7032cfd45a8a66538bbc598999270346b557cdf3d3da882b2daf094eec28"} Sep 15 12:15:20.238691 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.238675 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" event={"ID":"59fd9969-fdeb-4a2b-988e-650fb17da2cc","Type":"ContainerStarted","Data":"231a8ba910a0285cd456bf777380ea953ada3bab0f688ab018bc0d9b6b70c306"} Sep 15 12:15:20.265704 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.265643 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" podStartSLOduration=1.265627672 podStartE2EDuration="1.265627672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:20.263409573 +0000 UTC m=+772.020514645" watchObservedRunningTime="2026-09-15 12:15:20.265627672 +0000 UTC m=+772.022732739" Sep 15 12:15:20.299810 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.299765 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:20.404159 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.404121 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:20.409138 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.409109 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:20.592115 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.592049 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6"] Sep 15 12:15:20.594399 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:20.594374 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2736624f_181c_4b3b_9088_3b08bde5c003.slice/crio-36e6c743fbd329fbc807b4f3461f29e03f51486ae389869b9de4eabedc3253d9 WatchSource:0}: Error finding container 36e6c743fbd329fbc807b4f3461f29e03f51486ae389869b9de4eabedc3253d9: Status 404 returned error can't find the container with id 36e6c743fbd329fbc807b4f3461f29e03f51486ae389869b9de4eabedc3253d9 Sep 15 12:15:20.596760 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.596725 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:20.596870 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.596792 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:20.596870 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:20.596819 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:21.254226 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:21.254190 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" event={"ID":"2736624f-181c-4b3b-9088-3b08bde5c003","Type":"ContainerStarted","Data":"8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1"} Sep 15 12:15:21.254740 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:21.254273 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" event={"ID":"2736624f-181c-4b3b-9088-3b08bde5c003","Type":"ContainerStarted","Data":"36e6c743fbd329fbc807b4f3461f29e03f51486ae389869b9de4eabedc3253d9"} Sep 15 12:15:21.254740 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:21.254516 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:21.255472 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:21.255452 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:21.278234 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:21.278179 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" podStartSLOduration=2.278163593 podStartE2EDuration="2.278163593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:21.276160589 +0000 UTC m=+773.033265662" watchObservedRunningTime="2026-09-15 12:15:21.278163593 +0000 UTC m=+773.035268663" Sep 15 12:15:21.300784 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:21.300753 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:21.306644 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:21.306619 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:22.258940 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:22.258905 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:22.259809 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:22.259790 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:24.131108 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.131067 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k"] Sep 15 12:15:24.134615 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.134590 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:15:24.137397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.137377 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-4a9453\"" Sep 15 12:15:24.152301 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.152276 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k"] Sep 15 12:15:24.277799 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.277757 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04ab8ba0-ae23-4442-b8cc-5d02311a4228-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k\" (UID: \"04ab8ba0-ae23-4442-b8cc-5d02311a4228\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:15:24.278008 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.277844 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7sml\" (UniqueName: \"kubernetes.io/projected/04ab8ba0-ae23-4442-b8cc-5d02311a4228-kube-api-access-c7sml\") pod \"payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k\" (UID: \"04ab8ba0-ae23-4442-b8cc-5d02311a4228\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:15:24.379234 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.379187 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04ab8ba0-ae23-4442-b8cc-5d02311a4228-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k\" (UID: \"04ab8ba0-ae23-4442-b8cc-5d02311a4228\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:15:24.379545 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.379278 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7sml\" (UniqueName: \"kubernetes.io/projected/04ab8ba0-ae23-4442-b8cc-5d02311a4228-kube-api-access-c7sml\") pod \"payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k\" (UID: \"04ab8ba0-ae23-4442-b8cc-5d02311a4228\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:15:24.380652 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.380623 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04ab8ba0-ae23-4442-b8cc-5d02311a4228-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k\" (UID: \"04ab8ba0-ae23-4442-b8cc-5d02311a4228\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:15:24.388230 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.388162 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7sml\" (UniqueName: \"kubernetes.io/projected/04ab8ba0-ae23-4442-b8cc-5d02311a4228-kube-api-access-c7sml\") pod \"payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k\" (UID: \"04ab8ba0-ae23-4442-b8cc-5d02311a4228\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:15:24.445902 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.445871 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:15:24.743067 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:24.743045 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k"] Sep 15 12:15:24.745763 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:24.745735 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ab8ba0_ae23_4442_b8cc_5d02311a4228.slice/crio-a081cbb74cc20db60d2d0eb754f1d41adaa12e579b6600fee8c40bfd02514148 WatchSource:0}: Error finding container a081cbb74cc20db60d2d0eb754f1d41adaa12e579b6600fee8c40bfd02514148: Status 404 returned error can't find the container with id a081cbb74cc20db60d2d0eb754f1d41adaa12e579b6600fee8c40bfd02514148 Sep 15 12:15:25.279727 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:25.279687 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" event={"ID":"04ab8ba0-ae23-4442-b8cc-5d02311a4228","Type":"ContainerStarted","Data":"90168908313bafdac9203501703298335a3f61b8bb633851fe3158c6049c4234"} Sep 15 12:15:25.280142 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:25.279771 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" event={"ID":"04ab8ba0-ae23-4442-b8cc-5d02311a4228","Type":"ContainerStarted","Data":"a081cbb74cc20db60d2d0eb754f1d41adaa12e579b6600fee8c40bfd02514148"} Sep 15 12:15:25.280142 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:25.279837 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:15:25.321939 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:25.321854 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" podStartSLOduration=1.321823519 podStartE2EDuration="1.321823519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:25.313688453 +0000 UTC m=+777.070793525" watchObservedRunningTime="2026-09-15 12:15:25.321823519 +0000 UTC m=+777.078928675" Sep 15 12:15:26.022606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.022569 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj"] Sep 15 12:15:26.026394 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.026373 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:15:26.029010 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.028983 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-680229\"" Sep 15 12:15:26.029138 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.028983 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-680229-dockercfg-j9g7t\"" Sep 15 12:15:26.039444 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.039421 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj"] Sep 15 12:15:26.097933 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.097895 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppwbl\" (UniqueName: \"kubernetes.io/projected/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-kube-api-access-ppwbl\") pod \"payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj\" (UID: \"b8ad94c8-d45e-435b-aa5b-ead9d00f802e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:15:26.098126 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.098007 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj\" (UID: \"b8ad94c8-d45e-435b-aa5b-ead9d00f802e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:15:26.139773 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.139738 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn"] Sep 15 12:15:26.144555 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.144528 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:15:26.156174 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.156147 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn"] Sep 15 12:15:26.199568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.199532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3444551-04fb-482e-9f2f-5a74395afc19-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn\" (UID: \"b3444551-04fb-482e-9f2f-5a74395afc19\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:15:26.199749 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.199607 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppwbl\" (UniqueName: \"kubernetes.io/projected/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-kube-api-access-ppwbl\") pod \"payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj\" (UID: \"b8ad94c8-d45e-435b-aa5b-ead9d00f802e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:15:26.199749 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.199641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pjkn\" (UniqueName: \"kubernetes.io/projected/b3444551-04fb-482e-9f2f-5a74395afc19-kube-api-access-4pjkn\") pod \"payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn\" (UID: \"b3444551-04fb-482e-9f2f-5a74395afc19\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:15:26.199749 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.199722 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj\" (UID: \"b8ad94c8-d45e-435b-aa5b-ead9d00f802e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:15:26.200658 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.200635 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj\" (UID: \"b8ad94c8-d45e-435b-aa5b-ead9d00f802e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:15:26.209998 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.209970 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppwbl\" (UniqueName: \"kubernetes.io/projected/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-kube-api-access-ppwbl\") pod \"payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj\" (UID: \"b8ad94c8-d45e-435b-aa5b-ead9d00f802e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:15:26.300883 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.300792 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pjkn\" (UniqueName: \"kubernetes.io/projected/b3444551-04fb-482e-9f2f-5a74395afc19-kube-api-access-4pjkn\") pod \"payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn\" (UID: \"b3444551-04fb-482e-9f2f-5a74395afc19\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:15:26.301322 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.300914 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3444551-04fb-482e-9f2f-5a74395afc19-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn\" (UID: \"b3444551-04fb-482e-9f2f-5a74395afc19\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:15:26.301573 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.301554 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3444551-04fb-482e-9f2f-5a74395afc19-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn\" (UID: \"b3444551-04fb-482e-9f2f-5a74395afc19\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:15:26.311672 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.311650 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pjkn\" (UniqueName: \"kubernetes.io/projected/b3444551-04fb-482e-9f2f-5a74395afc19-kube-api-access-4pjkn\") pod \"payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn\" (UID: \"b3444551-04fb-482e-9f2f-5a74395afc19\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:15:26.336545 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.336513 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:15:26.457821 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.457785 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:15:26.614181 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.613835 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj"] Sep 15 12:15:26.617804 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:26.617700 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ad94c8_d45e_435b_aa5b_ead9d00f802e.slice/crio-edc480c2c5ecede848e0ad232c24a1b922c58382807acc65a7774ba046709d85 WatchSource:0}: Error finding container edc480c2c5ecede848e0ad232c24a1b922c58382807acc65a7774ba046709d85: Status 404 returned error can't find the container with id edc480c2c5ecede848e0ad232c24a1b922c58382807acc65a7774ba046709d85 Sep 15 12:15:26.744370 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:26.744273 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn"] Sep 15 12:15:26.747815 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:26.747788 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3444551_04fb_482e_9f2f_5a74395afc19.slice/crio-72a381142b21898f3da9f0f5c5e502ee4ac33cc0e77f0fa439332ef375d1c63a WatchSource:0}: Error finding container 72a381142b21898f3da9f0f5c5e502ee4ac33cc0e77f0fa439332ef375d1c63a: Status 404 returned error can't find the container with id 72a381142b21898f3da9f0f5c5e502ee4ac33cc0e77f0fa439332ef375d1c63a Sep 15 12:15:27.291521 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:27.291462 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" event={"ID":"b3444551-04fb-482e-9f2f-5a74395afc19","Type":"ContainerStarted","Data":"6afe172570d1ba53821f41a0fa87a9cba435c227972c1fad23a9aa28f28d8785"} Sep 15 12:15:27.291699 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:27.291546 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:15:27.291699 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:27.291568 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" event={"ID":"b3444551-04fb-482e-9f2f-5a74395afc19","Type":"ContainerStarted","Data":"72a381142b21898f3da9f0f5c5e502ee4ac33cc0e77f0fa439332ef375d1c63a"} Sep 15 12:15:27.293033 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:27.293008 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" event={"ID":"b8ad94c8-d45e-435b-aa5b-ead9d00f802e","Type":"ContainerStarted","Data":"41e18a24ff906e3105cfaab2191e032484b72b4a3394d01a2cec66811e4ad65d"} Sep 15 12:15:27.293141 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:27.293051 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" event={"ID":"b8ad94c8-d45e-435b-aa5b-ead9d00f802e","Type":"ContainerStarted","Data":"edc480c2c5ecede848e0ad232c24a1b922c58382807acc65a7774ba046709d85"} Sep 15 12:15:27.293141 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:27.293133 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:15:27.308744 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:27.307878 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" podStartSLOduration=1.307856289 podStartE2EDuration="1.307856289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:27.307848852 +0000 UTC m=+779.064953925" watchObservedRunningTime="2026-09-15 12:15:27.307856289 +0000 UTC m=+779.064961362" Sep 15 12:15:27.329212 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:27.329151 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" podStartSLOduration=1.329127637 podStartE2EDuration="1.329127637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:27.32594191 +0000 UTC m=+779.083046982" watchObservedRunningTime="2026-09-15 12:15:27.329127637 +0000 UTC m=+779.086232712" Sep 15 12:15:29.129915 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.129879 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j"] Sep 15 12:15:29.133166 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.133149 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:15:29.136086 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.136065 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-9ba48f\"" Sep 15 12:15:29.147245 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.147220 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j"] Sep 15 12:15:29.175025 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.174984 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz"] Sep 15 12:15:29.178301 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.178284 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:15:29.181178 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.181151 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-9ba48f-dockercfg-rrggm\"" Sep 15 12:15:29.193510 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.193466 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz"] Sep 15 12:15:29.239304 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.239267 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmwrk\" (UniqueName: \"kubernetes.io/projected/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-kube-api-access-jmwrk\") pod \"payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j\" (UID: \"c59d4179-f5d4-4175-83e4-ea3d4212a5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:15:29.239519 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.239375 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j\" (UID: \"c59d4179-f5d4-4175-83e4-ea3d4212a5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:15:29.340759 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.340726 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eec96e7-203a-47e1-9338-b7163ab056d8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz\" (UID: \"4eec96e7-203a-47e1-9338-b7163ab056d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:15:29.340924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.340786 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmwrk\" (UniqueName: \"kubernetes.io/projected/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-kube-api-access-jmwrk\") pod \"payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j\" (UID: \"c59d4179-f5d4-4175-83e4-ea3d4212a5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:15:29.340924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.340835 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48qpq\" (UniqueName: \"kubernetes.io/projected/4eec96e7-203a-47e1-9338-b7163ab056d8-kube-api-access-48qpq\") pod \"payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz\" (UID: \"4eec96e7-203a-47e1-9338-b7163ab056d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:15:29.340924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.340896 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j\" (UID: \"c59d4179-f5d4-4175-83e4-ea3d4212a5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:15:29.341609 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.341590 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j\" (UID: \"c59d4179-f5d4-4175-83e4-ea3d4212a5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:15:29.349413 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.349386 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmwrk\" (UniqueName: \"kubernetes.io/projected/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-kube-api-access-jmwrk\") pod \"payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j\" (UID: \"c59d4179-f5d4-4175-83e4-ea3d4212a5ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:15:29.442024 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.441920 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eec96e7-203a-47e1-9338-b7163ab056d8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz\" (UID: \"4eec96e7-203a-47e1-9338-b7163ab056d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:15:29.442024 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.442017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48qpq\" (UniqueName: \"kubernetes.io/projected/4eec96e7-203a-47e1-9338-b7163ab056d8-kube-api-access-48qpq\") pod \"payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz\" (UID: \"4eec96e7-203a-47e1-9338-b7163ab056d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:15:29.442653 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.442633 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eec96e7-203a-47e1-9338-b7163ab056d8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz\" (UID: \"4eec96e7-203a-47e1-9338-b7163ab056d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:15:29.443700 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.443682 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:15:29.453312 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.453286 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48qpq\" (UniqueName: \"kubernetes.io/projected/4eec96e7-203a-47e1-9338-b7163ab056d8-kube-api-access-48qpq\") pod \"payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz\" (UID: \"4eec96e7-203a-47e1-9338-b7163ab056d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:15:29.488478 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.488435 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:15:29.729842 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.729804 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j"] Sep 15 12:15:29.732172 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:29.732122 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59d4179_f5d4_4175_83e4_ea3d4212a5ef.slice/crio-9478d6e8b6a962575cbee026d28310578b6dff12f5de85704d582b44240d03e0 WatchSource:0}: Error finding container 9478d6e8b6a962575cbee026d28310578b6dff12f5de85704d582b44240d03e0: Status 404 returned error can't find the container with id 9478d6e8b6a962575cbee026d28310578b6dff12f5de85704d582b44240d03e0 Sep 15 12:15:29.784298 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:29.784274 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz"] Sep 15 12:15:29.787977 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:29.787944 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eec96e7_203a_47e1_9338_b7163ab056d8.slice/crio-e28f040b848a75a701419ba6d9dce25b30fed8998977d536cf6a04198aeb2f0f WatchSource:0}: Error finding container e28f040b848a75a701419ba6d9dce25b30fed8998977d536cf6a04198aeb2f0f: Status 404 returned error can't find the container with id e28f040b848a75a701419ba6d9dce25b30fed8998977d536cf6a04198aeb2f0f Sep 15 12:15:30.313530 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:30.313379 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" event={"ID":"4eec96e7-203a-47e1-9338-b7163ab056d8","Type":"ContainerStarted","Data":"17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8"} Sep 15 12:15:30.313530 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:30.313448 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:15:30.313530 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:30.313472 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" event={"ID":"4eec96e7-203a-47e1-9338-b7163ab056d8","Type":"ContainerStarted","Data":"e28f040b848a75a701419ba6d9dce25b30fed8998977d536cf6a04198aeb2f0f"} Sep 15 12:15:30.315806 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:30.315776 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" event={"ID":"c59d4179-f5d4-4175-83e4-ea3d4212a5ef","Type":"ContainerStarted","Data":"39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b"} Sep 15 12:15:30.315959 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:30.315830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" event={"ID":"c59d4179-f5d4-4175-83e4-ea3d4212a5ef","Type":"ContainerStarted","Data":"9478d6e8b6a962575cbee026d28310578b6dff12f5de85704d582b44240d03e0"} Sep 15 12:15:30.315959 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:30.315857 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:15:30.333275 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:30.333221 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" podStartSLOduration=1.333205444 podStartE2EDuration="1.333205444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:30.331057721 +0000 UTC m=+782.088162793" watchObservedRunningTime="2026-09-15 12:15:30.333205444 +0000 UTC m=+782.090310515" Sep 15 12:15:30.348284 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:30.348224 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" podStartSLOduration=1.348208858 podStartE2EDuration="1.348208858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:30.347125303 +0000 UTC m=+782.104230388" watchObservedRunningTime="2026-09-15 12:15:30.348208858 +0000 UTC m=+782.105313929" Sep 15 12:15:36.287262 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:36.287220 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:15:38.301681 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:38.300935 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:15:38.301681 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:38.301283 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:15:39.234057 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.234017 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr"] Sep 15 12:15:39.240262 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.240227 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:15:39.245038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.244511 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-0e10f0-dockercfg-d5rlg\"" Sep 15 12:15:39.245038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.244833 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-0e10f0\"" Sep 15 12:15:39.246628 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.246595 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr"] Sep 15 12:15:39.253929 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.253770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-plugins-config-volume\") pod \"payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr\" (UID: \"ec637855-ec5f-453f-9ea8-c5e9fec9cc27\") " pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:15:39.253929 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.253821 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v95nb\" (UniqueName: \"kubernetes.io/projected/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-kube-api-access-v95nb\") pod \"payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr\" (UID: \"ec637855-ec5f-453f-9ea8-c5e9fec9cc27\") " pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:15:39.354625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.354589 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-plugins-config-volume\") pod \"payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr\" (UID: \"ec637855-ec5f-453f-9ea8-c5e9fec9cc27\") " pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:15:39.355100 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.354642 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v95nb\" (UniqueName: \"kubernetes.io/projected/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-kube-api-access-v95nb\") pod \"payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr\" (UID: \"ec637855-ec5f-453f-9ea8-c5e9fec9cc27\") " pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:15:39.355409 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.355379 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-plugins-config-volume\") pod \"payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr\" (UID: \"ec637855-ec5f-453f-9ea8-c5e9fec9cc27\") " pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:15:39.366347 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.366309 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v95nb\" (UniqueName: \"kubernetes.io/projected/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-kube-api-access-v95nb\") pod \"payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr\" (UID: \"ec637855-ec5f-453f-9ea8-c5e9fec9cc27\") " pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:15:39.556836 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.556651 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:15:39.842086 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:39.842054 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr"] Sep 15 12:15:39.843175 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:39.843123 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec637855_ec5f_453f_9ea8_c5e9fec9cc27.slice/crio-1e03a3f3ca56b2e3e0214a653783ad28dde289d84583a28b78bf899f1712ce43 WatchSource:0}: Error finding container 1e03a3f3ca56b2e3e0214a653783ad28dde289d84583a28b78bf899f1712ce43: Status 404 returned error can't find the container with id 1e03a3f3ca56b2e3e0214a653783ad28dde289d84583a28b78bf899f1712ce43 Sep 15 12:15:40.420666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.420626 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" event={"ID":"ec637855-ec5f-453f-9ea8-c5e9fec9cc27","Type":"ContainerStarted","Data":"395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65"} Sep 15 12:15:40.421183 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.420702 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" event={"ID":"ec637855-ec5f-453f-9ea8-c5e9fec9cc27","Type":"ContainerStarted","Data":"1e03a3f3ca56b2e3e0214a653783ad28dde289d84583a28b78bf899f1712ce43"} Sep 15 12:15:40.421183 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.421033 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:15:40.431904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.431865 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk"] Sep 15 12:15:40.435113 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.435086 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:15:40.438696 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.438650 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-7d7eff3e\"" Sep 15 12:15:40.439524 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.439353 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-7d7eff3e-dockercfg-z2tvt\"" Sep 15 12:15:40.458034 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.457609 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" podStartSLOduration=1.457586168 podStartE2EDuration="1.457586168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:40.4513358 +0000 UTC m=+792.208440872" watchObservedRunningTime="2026-09-15 12:15:40.457586168 +0000 UTC m=+792.214691240" Sep 15 12:15:40.458034 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.457957 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk"] Sep 15 12:15:40.469351 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.467960 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwrk7\" (UniqueName: \"kubernetes.io/projected/46035c2c-b5e8-4b23-9b8b-d71807cb6091-kube-api-access-bwrk7\") pod \"payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk\" (UID: \"46035c2c-b5e8-4b23-9b8b-d71807cb6091\") " pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:15:40.469351 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.468118 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46035c2c-b5e8-4b23-9b8b-d71807cb6091-plugins-config-volume\") pod \"payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk\" (UID: \"46035c2c-b5e8-4b23-9b8b-d71807cb6091\") " pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:15:40.569597 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.569554 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46035c2c-b5e8-4b23-9b8b-d71807cb6091-plugins-config-volume\") pod \"payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk\" (UID: \"46035c2c-b5e8-4b23-9b8b-d71807cb6091\") " pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:15:40.569775 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.569720 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwrk7\" (UniqueName: \"kubernetes.io/projected/46035c2c-b5e8-4b23-9b8b-d71807cb6091-kube-api-access-bwrk7\") pod \"payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk\" (UID: \"46035c2c-b5e8-4b23-9b8b-d71807cb6091\") " pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:15:40.570452 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.570427 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46035c2c-b5e8-4b23-9b8b-d71807cb6091-plugins-config-volume\") pod \"payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk\" (UID: \"46035c2c-b5e8-4b23-9b8b-d71807cb6091\") " pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:15:40.579169 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.579140 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwrk7\" (UniqueName: \"kubernetes.io/projected/46035c2c-b5e8-4b23-9b8b-d71807cb6091-kube-api-access-bwrk7\") pod \"payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk\" (UID: \"46035c2c-b5e8-4b23-9b8b-d71807cb6091\") " pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:15:40.758291 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:40.758247 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:15:41.055051 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.055009 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk"] Sep 15 12:15:41.056478 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:41.056450 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46035c2c_b5e8_4b23_9b8b_d71807cb6091.slice/crio-27c14b3206011b00452448413df7c5502421ad2c234881036185f10f2350240b WatchSource:0}: Error finding container 27c14b3206011b00452448413df7c5502421ad2c234881036185f10f2350240b: Status 404 returned error can't find the container with id 27c14b3206011b00452448413df7c5502421ad2c234881036185f10f2350240b Sep 15 12:15:41.325918 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.325828 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:15:41.327275 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.326111 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:15:41.427317 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.427227 2582 generic.go:356] "Generic (PLEG): container finished" podID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerID="2acf7032cfd45a8a66538bbc598999270346b557cdf3d3da882b2daf094eec28" exitCode=0 Sep 15 12:15:41.427317 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.427298 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" event={"ID":"59fd9969-fdeb-4a2b-988e-650fb17da2cc","Type":"ContainerDied","Data":"2acf7032cfd45a8a66538bbc598999270346b557cdf3d3da882b2daf094eec28"} Sep 15 12:15:41.427832 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.427816 2582 scope.go:117] "RemoveContainer" containerID="2acf7032cfd45a8a66538bbc598999270346b557cdf3d3da882b2daf094eec28" Sep 15 12:15:41.429238 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.429198 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:41.429238 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.429221 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" event={"ID":"46035c2c-b5e8-4b23-9b8b-d71807cb6091","Type":"ContainerStarted","Data":"7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da"} Sep 15 12:15:41.429438 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.429268 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" event={"ID":"46035c2c-b5e8-4b23-9b8b-d71807cb6091","Type":"ContainerStarted","Data":"27c14b3206011b00452448413df7c5502421ad2c234881036185f10f2350240b"} Sep 15 12:15:41.429438 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.429278 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:41.429438 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.429296 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:15:41.429438 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.429326 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:41.431222 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.431197 2582 generic.go:356] "Generic (PLEG): container finished" podID="2736624f-181c-4b3b-9088-3b08bde5c003" containerID="8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1" exitCode=0 Sep 15 12:15:41.431317 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.431263 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" event={"ID":"2736624f-181c-4b3b-9088-3b08bde5c003","Type":"ContainerDied","Data":"8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1"} Sep 15 12:15:41.431642 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.431622 2582 scope.go:117] "RemoveContainer" containerID="8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1" Sep 15 12:15:41.439204 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.439165 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:41.439326 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.439242 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:41.439326 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.439274 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:15:41.466548 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:41.466474 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" podStartSLOduration=1.466453099 podStartE2EDuration="1.466453099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:41.464724836 +0000 UTC m=+793.221829907" watchObservedRunningTime="2026-09-15 12:15:41.466453099 +0000 UTC m=+793.223558172" Sep 15 12:15:42.442657 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:42.442612 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" event={"ID":"59fd9969-fdeb-4a2b-988e-650fb17da2cc","Type":"ContainerStarted","Data":"206c5dc76f2d60bc3267978c49e69d52eaff6df1c2c2068c4d4517737547a734"} Sep 15 12:15:42.448030 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:42.447994 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" event={"ID":"2736624f-181c-4b3b-9088-3b08bde5c003","Type":"ContainerStarted","Data":"14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd"} Sep 15 12:15:43.300811 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:43.300753 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:43.301108 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:43.301078 2582 patch_prober.go:28] interesting pod/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Sep 15 12:15:43.301228 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:43.301149 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Sep 15 12:15:43.404500 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:43.404438 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:43.405018 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:43.404971 2582 patch_prober.go:28] interesting pod/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Sep 15 12:15:43.405276 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:43.405235 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Sep 15 12:15:44.301197 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:44.301160 2582 patch_prober.go:28] interesting pod/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Sep 15 12:15:44.301820 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:44.301786 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Sep 15 12:15:44.404772 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:44.404732 2582 patch_prober.go:28] interesting pod/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Sep 15 12:15:44.404978 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:44.404804 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Sep 15 12:15:45.301222 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:45.301173 2582 patch_prober.go:28] interesting pod/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Sep 15 12:15:45.301731 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:45.301246 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Sep 15 12:15:45.403869 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:45.403829 2582 patch_prober.go:28] interesting pod/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Sep 15 12:15:45.404166 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:45.404134 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Sep 15 12:15:46.300996 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:46.300951 2582 patch_prober.go:28] interesting pod/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Sep 15 12:15:46.301224 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:46.301021 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Sep 15 12:15:46.404883 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:46.404839 2582 patch_prober.go:28] interesting pod/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Sep 15 12:15:46.405392 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:46.404934 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Sep 15 12:15:47.301033 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:47.300978 2582 patch_prober.go:28] interesting pod/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Sep 15 12:15:47.301275 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:47.301055 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Sep 15 12:15:47.404416 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:47.404367 2582 patch_prober.go:28] interesting pod/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Sep 15 12:15:47.404768 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:47.404719 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Sep 15 12:15:48.304351 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:48.304317 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:48.304780 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:48.304589 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:48.305414 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:48.305395 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:15:48.409097 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:48.409066 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:48.409507 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:48.409463 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:48.410360 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:48.410340 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:15:49.432583 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.432543 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p"] Sep 15 12:15:49.437207 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.437174 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:15:49.440804 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.440763 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-69d7ba\"" Sep 15 12:15:49.440946 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.440922 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-69d7ba-dockercfg-4rwxh\"" Sep 15 12:15:49.446551 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.446225 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p"] Sep 15 12:15:49.484255 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.484222 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq8pg\" (UniqueName: \"kubernetes.io/projected/8e017f15-2533-4da1-83ce-0fc49d8c832f-kube-api-access-qq8pg\") pod \"payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p\" (UID: \"8e017f15-2533-4da1-83ce-0fc49d8c832f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:15:49.484457 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.484277 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e017f15-2533-4da1-83ce-0fc49d8c832f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p\" (UID: \"8e017f15-2533-4da1-83ce-0fc49d8c832f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:15:49.585175 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.585134 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq8pg\" (UniqueName: \"kubernetes.io/projected/8e017f15-2533-4da1-83ce-0fc49d8c832f-kube-api-access-qq8pg\") pod \"payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p\" (UID: \"8e017f15-2533-4da1-83ce-0fc49d8c832f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:15:49.585402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.585297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e017f15-2533-4da1-83ce-0fc49d8c832f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p\" (UID: \"8e017f15-2533-4da1-83ce-0fc49d8c832f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:15:49.586121 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.586098 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e017f15-2533-4da1-83ce-0fc49d8c832f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p\" (UID: \"8e017f15-2533-4da1-83ce-0fc49d8c832f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:15:49.593923 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.593896 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq8pg\" (UniqueName: \"kubernetes.io/projected/8e017f15-2533-4da1-83ce-0fc49d8c832f-kube-api-access-qq8pg\") pod \"payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p\" (UID: \"8e017f15-2533-4da1-83ce-0fc49d8c832f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:15:49.751534 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:49.751499 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:15:50.023809 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:50.023783 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p"] Sep 15 12:15:50.025637 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:15:50.025613 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e017f15_2533_4da1_83ce_0fc49d8c832f.slice/crio-55c42c715f9eb54399f8560abb5b3e991545521d28fca588c25d27343aa28e2f WatchSource:0}: Error finding container 55c42c715f9eb54399f8560abb5b3e991545521d28fca588c25d27343aa28e2f: Status 404 returned error can't find the container with id 55c42c715f9eb54399f8560abb5b3e991545521d28fca588c25d27343aa28e2f Sep 15 12:15:50.500153 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:50.500113 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" event={"ID":"8e017f15-2533-4da1-83ce-0fc49d8c832f","Type":"ContainerStarted","Data":"99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee"} Sep 15 12:15:50.500604 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:50.500167 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" event={"ID":"8e017f15-2533-4da1-83ce-0fc49d8c832f","Type":"ContainerStarted","Data":"55c42c715f9eb54399f8560abb5b3e991545521d28fca588c25d27343aa28e2f"} Sep 15 12:15:50.500604 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:50.500239 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:15:50.522556 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:50.522505 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" podStartSLOduration=1.522475777 podStartE2EDuration="1.522475777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:15:50.520368314 +0000 UTC m=+802.277473376" watchObservedRunningTime="2026-09-15 12:15:50.522475777 +0000 UTC m=+802.279580847" Sep 15 12:15:51.433597 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:51.433566 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:15:52.449923 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:15:52.449893 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:16:01.511474 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:01.511444 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:16:17.091225 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.091177 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk"] Sep 15 12:16:17.091860 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.091446 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" podUID="46035c2c-b5e8-4b23-9b8b-d71807cb6091" containerName="payload-processing" containerID="cri-o://7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da" gracePeriod=30 Sep 15 12:16:17.348072 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.348007 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:16:17.374565 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.374535 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwrk7\" (UniqueName: \"kubernetes.io/projected/46035c2c-b5e8-4b23-9b8b-d71807cb6091-kube-api-access-bwrk7\") pod \"46035c2c-b5e8-4b23-9b8b-d71807cb6091\" (UID: \"46035c2c-b5e8-4b23-9b8b-d71807cb6091\") " Sep 15 12:16:17.374743 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.374587 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46035c2c-b5e8-4b23-9b8b-d71807cb6091-plugins-config-volume\") pod \"46035c2c-b5e8-4b23-9b8b-d71807cb6091\" (UID: \"46035c2c-b5e8-4b23-9b8b-d71807cb6091\") " Sep 15 12:16:17.376108 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.375323 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46035c2c-b5e8-4b23-9b8b-d71807cb6091-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "46035c2c-b5e8-4b23-9b8b-d71807cb6091" (UID: "46035c2c-b5e8-4b23-9b8b-d71807cb6091"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:16:17.384084 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.383474 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46035c2c-b5e8-4b23-9b8b-d71807cb6091-kube-api-access-bwrk7" (OuterVolumeSpecName: "kube-api-access-bwrk7") pod "46035c2c-b5e8-4b23-9b8b-d71807cb6091" (UID: "46035c2c-b5e8-4b23-9b8b-d71807cb6091"). InnerVolumeSpecName "kube-api-access-bwrk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:16:17.384084 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.384015 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwrk7\" (UniqueName: \"kubernetes.io/projected/46035c2c-b5e8-4b23-9b8b-d71807cb6091-kube-api-access-bwrk7\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:16:17.384084 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.384060 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46035c2c-b5e8-4b23-9b8b-d71807cb6091-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:16:17.650909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.650814 2582 generic.go:356] "Generic (PLEG): container finished" podID="46035c2c-b5e8-4b23-9b8b-d71807cb6091" containerID="7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da" exitCode=0 Sep 15 12:16:17.650909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.650886 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" Sep 15 12:16:17.651091 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.650906 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" event={"ID":"46035c2c-b5e8-4b23-9b8b-d71807cb6091","Type":"ContainerDied","Data":"7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da"} Sep 15 12:16:17.651091 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.650991 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk" event={"ID":"46035c2c-b5e8-4b23-9b8b-d71807cb6091","Type":"ContainerDied","Data":"27c14b3206011b00452448413df7c5502421ad2c234881036185f10f2350240b"} Sep 15 12:16:17.651091 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.651015 2582 scope.go:117] "RemoveContainer" containerID="7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da" Sep 15 12:16:17.660504 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.660470 2582 scope.go:117] "RemoveContainer" containerID="7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da" Sep 15 12:16:17.660858 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:16:17.660839 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da\": container with ID starting with 7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da not found: ID does not exist" containerID="7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da" Sep 15 12:16:17.660945 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.660872 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da"} err="failed to get container status \"7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da\": rpc error: code = NotFound desc = could not find container \"7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da\": container with ID starting with 7961f3712d54683a76b400da82e2c8fcd59b3e0c354ea546fadfcece31d573da not found: ID does not exist" Sep 15 12:16:17.675213 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.675180 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk"] Sep 15 12:16:17.677173 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:17.677150 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7d7eff3e-7c57776d9c-fx9jk"] Sep 15 12:16:18.712065 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:18.712027 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46035c2c-b5e8-4b23-9b8b-d71807cb6091" path="/var/lib/kubelet/pods/46035c2c-b5e8-4b23-9b8b-d71807cb6091/volumes" Sep 15 12:16:46.291375 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.291341 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8"] Sep 15 12:16:46.291814 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.291799 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46035c2c-b5e8-4b23-9b8b-d71807cb6091" containerName="payload-processing" Sep 15 12:16:46.291814 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.291815 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="46035c2c-b5e8-4b23-9b8b-d71807cb6091" containerName="payload-processing" Sep 15 12:16:46.291893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.291885 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="46035c2c-b5e8-4b23-9b8b-d71807cb6091" containerName="payload-processing" Sep 15 12:16:46.294685 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.294665 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8" Sep 15 12:16:46.297373 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.297348 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:16:46.297523 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.297419 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:16:46.304325 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.304304 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8"] Sep 15 12:16:46.378021 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.377981 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/420343f5-bb8e-4a96-88fc-afb67d718bac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8\" (UID: \"420343f5-bb8e-4a96-88fc-afb67d718bac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8" Sep 15 12:16:46.479360 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.479326 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/420343f5-bb8e-4a96-88fc-afb67d718bac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8\" (UID: \"420343f5-bb8e-4a96-88fc-afb67d718bac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8" Sep 15 12:16:46.479977 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.479956 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/420343f5-bb8e-4a96-88fc-afb67d718bac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8\" (UID: \"420343f5-bb8e-4a96-88fc-afb67d718bac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8" Sep 15 12:16:46.606281 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.606196 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8" Sep 15 12:16:46.880626 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:46.880550 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8"] Sep 15 12:16:46.885178 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:16:46.885150 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420343f5_bb8e_4a96_88fc_afb67d718bac.slice/crio-cbea9704c7937ebac5a6b787c262c2f5d3dbf04907f09fd66cf6ff030ed757e7 WatchSource:0}: Error finding container cbea9704c7937ebac5a6b787c262c2f5d3dbf04907f09fd66cf6ff030ed757e7: Status 404 returned error can't find the container with id cbea9704c7937ebac5a6b787c262c2f5d3dbf04907f09fd66cf6ff030ed757e7 Sep 15 12:16:47.811336 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:47.811298 2582 generic.go:356] "Generic (PLEG): container finished" podID="420343f5-bb8e-4a96-88fc-afb67d718bac" containerID="7d2b78868ba65c84c4340f491e25162d4304403a4eca5ed26e926331e496b6bf" exitCode=0 Sep 15 12:16:47.811779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:47.811344 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8" event={"ID":"420343f5-bb8e-4a96-88fc-afb67d718bac","Type":"ContainerDied","Data":"7d2b78868ba65c84c4340f491e25162d4304403a4eca5ed26e926331e496b6bf"} Sep 15 12:16:47.811779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:47.811400 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8" event={"ID":"420343f5-bb8e-4a96-88fc-afb67d718bac","Type":"ContainerStarted","Data":"cbea9704c7937ebac5a6b787c262c2f5d3dbf04907f09fd66cf6ff030ed757e7"} Sep 15 12:16:48.955643 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:48.955620 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8" Sep 15 12:16:49.005866 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:49.005834 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/420343f5-bb8e-4a96-88fc-afb67d718bac-maas-api-service-ca\") pod \"420343f5-bb8e-4a96-88fc-afb67d718bac\" (UID: \"420343f5-bb8e-4a96-88fc-afb67d718bac\") " Sep 15 12:16:49.006198 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:49.006173 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/420343f5-bb8e-4a96-88fc-afb67d718bac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "420343f5-bb8e-4a96-88fc-afb67d718bac" (UID: "420343f5-bb8e-4a96-88fc-afb67d718bac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:16:49.106988 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:49.106897 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/420343f5-bb8e-4a96-88fc-afb67d718bac-maas-api-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:16:49.822373 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:49.822344 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8" Sep 15 12:16:49.822573 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:49.822343 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8" event={"ID":"420343f5-bb8e-4a96-88fc-afb67d718bac","Type":"ContainerDied","Data":"cbea9704c7937ebac5a6b787c262c2f5d3dbf04907f09fd66cf6ff030ed757e7"} Sep 15 12:16:49.822573 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:49.822465 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbea9704c7937ebac5a6b787c262c2f5d3dbf04907f09fd66cf6ff030ed757e7" Sep 15 12:16:53.314548 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:53.314517 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8"] Sep 15 12:16:53.316836 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:53.316800 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1a5ccf40-l6nj8"] Sep 15 12:16:54.712018 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:16:54.711989 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="420343f5-bb8e-4a96-88fc-afb67d718bac" path="/var/lib/kubelet/pods/420343f5-bb8e-4a96-88fc-afb67d718bac/volumes" Sep 15 12:17:35.141472 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.141435 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr"] Sep 15 12:17:35.141927 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.141914 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="420343f5-bb8e-4a96-88fc-afb67d718bac" containerName="revoke-keys" Sep 15 12:17:35.141980 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.141929 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="420343f5-bb8e-4a96-88fc-afb67d718bac" containerName="revoke-keys" Sep 15 12:17:35.142015 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.142008 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="420343f5-bb8e-4a96-88fc-afb67d718bac" containerName="revoke-keys" Sep 15 12:17:35.143961 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.143945 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:35.150060 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.150037 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-66020bb6\"" Sep 15 12:17:35.150212 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.150150 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-66020bb6-dockercfg-nltf2\"" Sep 15 12:17:35.159811 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.159783 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr"] Sep 15 12:17:35.274629 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.274584 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-plugins-config-volume\") pod \"payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr\" (UID: \"bf574d88-b1ee-4c14-a7ec-7e17608cdf74\") " pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:35.274629 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.274632 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nztv\" (UniqueName: \"kubernetes.io/projected/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-kube-api-access-2nztv\") pod \"payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr\" (UID: \"bf574d88-b1ee-4c14-a7ec-7e17608cdf74\") " pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:35.375967 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.375921 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-plugins-config-volume\") pod \"payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr\" (UID: \"bf574d88-b1ee-4c14-a7ec-7e17608cdf74\") " pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:35.375967 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.375966 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nztv\" (UniqueName: \"kubernetes.io/projected/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-kube-api-access-2nztv\") pod \"payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr\" (UID: \"bf574d88-b1ee-4c14-a7ec-7e17608cdf74\") " pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:35.376748 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.376715 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-plugins-config-volume\") pod \"payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr\" (UID: \"bf574d88-b1ee-4c14-a7ec-7e17608cdf74\") " pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:35.386945 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.386917 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nztv\" (UniqueName: \"kubernetes.io/projected/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-kube-api-access-2nztv\") pod \"payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr\" (UID: \"bf574d88-b1ee-4c14-a7ec-7e17608cdf74\") " pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:35.454365 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.454272 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:35.736075 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:35.736050 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr"] Sep 15 12:17:35.738406 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:17:35.738383 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf574d88_b1ee_4c14_a7ec_7e17608cdf74.slice/crio-b59d26b8de8f3a5fc5675dc57aa1245108e357f58e34533eaab233f80699d2a6 WatchSource:0}: Error finding container b59d26b8de8f3a5fc5675dc57aa1245108e357f58e34533eaab233f80699d2a6: Status 404 returned error can't find the container with id b59d26b8de8f3a5fc5675dc57aa1245108e357f58e34533eaab233f80699d2a6 Sep 15 12:17:36.076931 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:36.076829 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" event={"ID":"bf574d88-b1ee-4c14-a7ec-7e17608cdf74","Type":"ContainerStarted","Data":"1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316"} Sep 15 12:17:36.076931 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:36.076887 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" event={"ID":"bf574d88-b1ee-4c14-a7ec-7e17608cdf74","Type":"ContainerStarted","Data":"b59d26b8de8f3a5fc5675dc57aa1245108e357f58e34533eaab233f80699d2a6"} Sep 15 12:17:36.077213 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:36.076928 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:36.095642 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:36.095585 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" podStartSLOduration=1.095563653 podStartE2EDuration="1.095563653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:17:36.095304575 +0000 UTC m=+907.852409646" watchObservedRunningTime="2026-09-15 12:17:36.095563653 +0000 UTC m=+907.852668726" Sep 15 12:17:47.085714 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:47.085680 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:54.177063 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:54.177031 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr"] Sep 15 12:17:54.177636 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:54.177251 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" podUID="ec637855-ec5f-453f-9ea8-c5e9fec9cc27" containerName="payload-processing" containerID="cri-o://395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65" gracePeriod=30 Sep 15 12:17:54.443022 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:54.442947 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:17:54.481678 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:54.481638 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v95nb\" (UniqueName: \"kubernetes.io/projected/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-kube-api-access-v95nb\") pod \"ec637855-ec5f-453f-9ea8-c5e9fec9cc27\" (UID: \"ec637855-ec5f-453f-9ea8-c5e9fec9cc27\") " Sep 15 12:17:54.481851 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:54.481808 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-plugins-config-volume\") pod \"ec637855-ec5f-453f-9ea8-c5e9fec9cc27\" (UID: \"ec637855-ec5f-453f-9ea8-c5e9fec9cc27\") " Sep 15 12:17:54.482160 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:54.482132 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec637855-ec5f-453f-9ea8-c5e9fec9cc27" (UID: "ec637855-ec5f-453f-9ea8-c5e9fec9cc27"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:17:54.483987 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:54.483958 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-kube-api-access-v95nb" (OuterVolumeSpecName: "kube-api-access-v95nb") pod "ec637855-ec5f-453f-9ea8-c5e9fec9cc27" (UID: "ec637855-ec5f-453f-9ea8-c5e9fec9cc27"). InnerVolumeSpecName "kube-api-access-v95nb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:17:54.583446 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:54.583409 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v95nb\" (UniqueName: \"kubernetes.io/projected/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-kube-api-access-v95nb\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:17:54.583446 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:54.583442 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec637855-ec5f-453f-9ea8-c5e9fec9cc27-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:17:55.187277 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:55.187182 2582 generic.go:356] "Generic (PLEG): container finished" podID="ec637855-ec5f-453f-9ea8-c5e9fec9cc27" containerID="395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65" exitCode=0 Sep 15 12:17:55.187277 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:55.187226 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" Sep 15 12:17:55.187277 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:55.187263 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" event={"ID":"ec637855-ec5f-453f-9ea8-c5e9fec9cc27","Type":"ContainerDied","Data":"395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65"} Sep 15 12:17:55.187843 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:55.187316 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr" event={"ID":"ec637855-ec5f-453f-9ea8-c5e9fec9cc27","Type":"ContainerDied","Data":"1e03a3f3ca56b2e3e0214a653783ad28dde289d84583a28b78bf899f1712ce43"} Sep 15 12:17:55.187843 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:55.187345 2582 scope.go:117] "RemoveContainer" containerID="395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65" Sep 15 12:17:55.196564 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:55.196546 2582 scope.go:117] "RemoveContainer" containerID="395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65" Sep 15 12:17:55.196874 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:17:55.196856 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65\": container with ID starting with 395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65 not found: ID does not exist" containerID="395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65" Sep 15 12:17:55.196927 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:55.196884 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65"} err="failed to get container status \"395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65\": rpc error: code = NotFound desc = could not find container \"395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65\": container with ID starting with 395b8bcd72993e24b6d32445808deb2e42caafa33fb8fd4fccdb9fb3ad089a65 not found: ID does not exist" Sep 15 12:17:55.208201 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:55.208171 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr"] Sep 15 12:17:55.215391 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:55.215365 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0e10f0-67b58d54f7-7h8nr"] Sep 15 12:17:56.711774 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:56.711736 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec637855-ec5f-453f-9ea8-c5e9fec9cc27" path="/var/lib/kubelet/pods/ec637855-ec5f-453f-9ea8-c5e9fec9cc27/volumes" Sep 15 12:17:58.920794 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:58.920760 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr"] Sep 15 12:17:58.921280 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:58.921018 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" podUID="bf574d88-b1ee-4c14-a7ec-7e17608cdf74" containerName="payload-processing" containerID="cri-o://1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316" gracePeriod=30 Sep 15 12:17:59.180463 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.180402 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:59.209963 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.209932 2582 generic.go:356] "Generic (PLEG): container finished" podID="bf574d88-b1ee-4c14-a7ec-7e17608cdf74" containerID="1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316" exitCode=0 Sep 15 12:17:59.210138 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.209995 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" Sep 15 12:17:59.210138 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.209994 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" event={"ID":"bf574d88-b1ee-4c14-a7ec-7e17608cdf74","Type":"ContainerDied","Data":"1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316"} Sep 15 12:17:59.210138 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.210123 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr" event={"ID":"bf574d88-b1ee-4c14-a7ec-7e17608cdf74","Type":"ContainerDied","Data":"b59d26b8de8f3a5fc5675dc57aa1245108e357f58e34533eaab233f80699d2a6"} Sep 15 12:17:59.210260 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.210146 2582 scope.go:117] "RemoveContainer" containerID="1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316" Sep 15 12:17:59.224301 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.224172 2582 scope.go:117] "RemoveContainer" containerID="1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316" Sep 15 12:17:59.225745 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:17:59.225690 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316\": container with ID starting with 1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316 not found: ID does not exist" containerID="1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316" Sep 15 12:17:59.225917 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.225749 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316"} err="failed to get container status \"1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316\": rpc error: code = NotFound desc = could not find container \"1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316\": container with ID starting with 1ec64977e3083fa37664226c10ad84942cad275ae043cff25e5bdeb0547d8316 not found: ID does not exist" Sep 15 12:17:59.234735 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.233863 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-plugins-config-volume\") pod \"bf574d88-b1ee-4c14-a7ec-7e17608cdf74\" (UID: \"bf574d88-b1ee-4c14-a7ec-7e17608cdf74\") " Sep 15 12:17:59.234735 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.233955 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nztv\" (UniqueName: \"kubernetes.io/projected/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-kube-api-access-2nztv\") pod \"bf574d88-b1ee-4c14-a7ec-7e17608cdf74\" (UID: \"bf574d88-b1ee-4c14-a7ec-7e17608cdf74\") " Sep 15 12:17:59.235450 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.235390 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf574d88-b1ee-4c14-a7ec-7e17608cdf74" (UID: "bf574d88-b1ee-4c14-a7ec-7e17608cdf74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:17:59.240545 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.240460 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-kube-api-access-2nztv" (OuterVolumeSpecName: "kube-api-access-2nztv") pod "bf574d88-b1ee-4c14-a7ec-7e17608cdf74" (UID: "bf574d88-b1ee-4c14-a7ec-7e17608cdf74"). InnerVolumeSpecName "kube-api-access-2nztv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:17:59.335297 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.335257 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:17:59.335297 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.335289 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nztv\" (UniqueName: \"kubernetes.io/projected/bf574d88-b1ee-4c14-a7ec-7e17608cdf74-kube-api-access-2nztv\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:17:59.535494 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.535448 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr"] Sep 15 12:17:59.543875 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:17:59.543845 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-66020bb6-7d4fc58d89-lmwtr"] Sep 15 12:18:00.711908 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:18:00.711874 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf574d88-b1ee-4c14-a7ec-7e17608cdf74" path="/var/lib/kubelet/pods/bf574d88-b1ee-4c14-a7ec-7e17608cdf74/volumes" Sep 15 12:19:40.353329 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.353273 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw"] Sep 15 12:19:40.357734 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.355067 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec637855-ec5f-453f-9ea8-c5e9fec9cc27" containerName="payload-processing" Sep 15 12:19:40.357734 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.355118 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec637855-ec5f-453f-9ea8-c5e9fec9cc27" containerName="payload-processing" Sep 15 12:19:40.357734 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.355167 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf574d88-b1ee-4c14-a7ec-7e17608cdf74" containerName="payload-processing" Sep 15 12:19:40.357734 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.355197 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf574d88-b1ee-4c14-a7ec-7e17608cdf74" containerName="payload-processing" Sep 15 12:19:40.357734 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.355417 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf574d88-b1ee-4c14-a7ec-7e17608cdf74" containerName="payload-processing" Sep 15 12:19:40.357734 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.355439 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec637855-ec5f-453f-9ea8-c5e9fec9cc27" containerName="payload-processing" Sep 15 12:19:40.364825 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.364785 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.369203 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.369181 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-167943df-openshift-default-dockercfg-gj8wg\"" Sep 15 12:19:40.392403 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.392371 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw"] Sep 15 12:19:40.482977 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.482941 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-credential-socket\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.483170 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.483014 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-token\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.483170 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.483057 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-podinfo\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.483170 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.483083 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-certs\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.483170 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.483112 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istiod-ca-cert\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.483170 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.483146 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-data\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.483381 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.483240 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtld6\" (UniqueName: \"kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-kube-api-access-rtld6\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.483381 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.483286 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-envoy\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.483381 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.483330 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-socket\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.584435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.584400 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-data\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.584631 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.584451 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtld6\" (UniqueName: \"kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-kube-api-access-rtld6\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.584631 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.584568 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-envoy\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.584631 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.584612 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-socket\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.584802 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.584664 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-credential-socket\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.584802 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.584718 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-token\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.584802 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.584743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-podinfo\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.584802 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.584767 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-certs\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.584802 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.584789 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istiod-ca-cert\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.585068 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.584865 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-data\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.585068 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.585041 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-socket\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.585171 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.585130 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-certs\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.585171 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.585130 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-credential-socket\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.585467 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.585439 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istiod-ca-cert\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.586976 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.586957 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-envoy\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.587511 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.587474 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-podinfo\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.615599 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.615529 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-token\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.615725 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.615603 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtld6\" (UniqueName: \"kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-kube-api-access-rtld6\") pod \"e2e-mt-167943df-openshift-default-974794956-4jxrw\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.676061 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.676029 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:40.968509 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.966795 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw"] Sep 15 12:19:40.970519 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:19:40.970472 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb555b01f_ca2b_4bfc_a6e8_90d856ca6bc6.slice/crio-4f7bb285f9cb1b1c07921b7d7420a4391f220657bd3ea41db299cd6e1814f34d WatchSource:0}: Error finding container 4f7bb285f9cb1b1c07921b7d7420a4391f220657bd3ea41db299cd6e1814f34d: Status 404 returned error can't find the container with id 4f7bb285f9cb1b1c07921b7d7420a4391f220657bd3ea41db299cd6e1814f34d Sep 15 12:19:40.972721 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.972673 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:19:40.972820 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.972754 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:19:40.972820 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:40.972785 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:19:41.754433 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:41.754398 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" event={"ID":"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6","Type":"ContainerStarted","Data":"7509c7a894dcd48a797f941e90ad93353d1bccbd2670418d52588d096e46bd77"} Sep 15 12:19:41.754840 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:41.754449 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" event={"ID":"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6","Type":"ContainerStarted","Data":"4f7bb285f9cb1b1c07921b7d7420a4391f220657bd3ea41db299cd6e1814f34d"} Sep 15 12:19:41.793580 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:41.793519 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" podStartSLOduration=1.7934984360000001 podStartE2EDuration="1.793498436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:19:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:19:41.793267555 +0000 UTC m=+1033.550372627" watchObservedRunningTime="2026-09-15 12:19:41.793498436 +0000 UTC m=+1033.550603499" Sep 15 12:19:42.677237 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:42.677196 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:42.682424 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:42.682394 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:42.759336 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:42.759300 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:42.760296 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:42.760269 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:19:53.587681 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.587650 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch"] Sep 15 12:19:53.591165 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.591147 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:19:53.596030 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.596005 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-167943df-dockercfg-gt4fs\"" Sep 15 12:19:53.596159 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.596080 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-167943df\"" Sep 15 12:19:53.611173 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.611144 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch"] Sep 15 12:19:53.689111 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.689078 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk"] Sep 15 12:19:53.692659 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.692642 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:19:53.711786 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.711761 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk"] Sep 15 12:19:53.726419 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.726385 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9868c351-9407-47ef-a662-fedc60f9d016-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch\" (UID: \"9868c351-9407-47ef-a662-fedc60f9d016\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:19:53.726581 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.726453 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv9gg\" (UniqueName: \"kubernetes.io/projected/9868c351-9407-47ef-a662-fedc60f9d016-kube-api-access-lv9gg\") pod \"payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch\" (UID: \"9868c351-9407-47ef-a662-fedc60f9d016\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:19:53.827113 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.827080 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9868c351-9407-47ef-a662-fedc60f9d016-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch\" (UID: \"9868c351-9407-47ef-a662-fedc60f9d016\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:19:53.827322 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.827140 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c8dd440-864b-4334-99c5-bc671ae88b01-plugins-config-volume\") pod \"payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk\" (UID: \"4c8dd440-864b-4334-99c5-bc671ae88b01\") " pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:19:53.827322 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.827200 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtv4m\" (UniqueName: \"kubernetes.io/projected/4c8dd440-864b-4334-99c5-bc671ae88b01-kube-api-access-jtv4m\") pod \"payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk\" (UID: \"4c8dd440-864b-4334-99c5-bc671ae88b01\") " pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:19:53.827322 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.827248 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv9gg\" (UniqueName: \"kubernetes.io/projected/9868c351-9407-47ef-a662-fedc60f9d016-kube-api-access-lv9gg\") pod \"payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch\" (UID: \"9868c351-9407-47ef-a662-fedc60f9d016\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:19:53.827904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.827883 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9868c351-9407-47ef-a662-fedc60f9d016-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch\" (UID: \"9868c351-9407-47ef-a662-fedc60f9d016\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:19:53.836667 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.836636 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv9gg\" (UniqueName: \"kubernetes.io/projected/9868c351-9407-47ef-a662-fedc60f9d016-kube-api-access-lv9gg\") pod \"payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch\" (UID: \"9868c351-9407-47ef-a662-fedc60f9d016\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:19:53.902283 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.902188 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:19:53.928536 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.928502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c8dd440-864b-4334-99c5-bc671ae88b01-plugins-config-volume\") pod \"payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk\" (UID: \"4c8dd440-864b-4334-99c5-bc671ae88b01\") " pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:19:53.928698 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.928563 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtv4m\" (UniqueName: \"kubernetes.io/projected/4c8dd440-864b-4334-99c5-bc671ae88b01-kube-api-access-jtv4m\") pod \"payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk\" (UID: \"4c8dd440-864b-4334-99c5-bc671ae88b01\") " pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:19:53.929219 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.929195 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c8dd440-864b-4334-99c5-bc671ae88b01-plugins-config-volume\") pod \"payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk\" (UID: \"4c8dd440-864b-4334-99c5-bc671ae88b01\") " pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:19:53.941089 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:53.941054 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtv4m\" (UniqueName: \"kubernetes.io/projected/4c8dd440-864b-4334-99c5-bc671ae88b01-kube-api-access-jtv4m\") pod \"payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk\" (UID: \"4c8dd440-864b-4334-99c5-bc671ae88b01\") " pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:19:54.003378 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.003345 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:19:54.182209 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.182174 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch"] Sep 15 12:19:54.183516 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:19:54.183475 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9868c351_9407_47ef_a662_fedc60f9d016.slice/crio-c610857c986471494ffb5822d44558583165f1a31e22fd8d6a2cf835fabb0c57 WatchSource:0}: Error finding container c610857c986471494ffb5822d44558583165f1a31e22fd8d6a2cf835fabb0c57: Status 404 returned error can't find the container with id c610857c986471494ffb5822d44558583165f1a31e22fd8d6a2cf835fabb0c57 Sep 15 12:19:54.283652 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.283624 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk"] Sep 15 12:19:54.285393 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:19:54.285365 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8dd440_864b_4334_99c5_bc671ae88b01.slice/crio-c0e30e5ad6d44a8767946a18e63258341266eb3017d47385a2f3edc1e93cb97a WatchSource:0}: Error finding container c0e30e5ad6d44a8767946a18e63258341266eb3017d47385a2f3edc1e93cb97a: Status 404 returned error can't find the container with id c0e30e5ad6d44a8767946a18e63258341266eb3017d47385a2f3edc1e93cb97a Sep 15 12:19:54.831435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.831392 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" event={"ID":"4c8dd440-864b-4334-99c5-bc671ae88b01","Type":"ContainerStarted","Data":"3de4b89aa40a01eb5e2d0f84a263fd37ef4068c414020a88857dcbf6346ff267"} Sep 15 12:19:54.831935 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.831455 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" event={"ID":"4c8dd440-864b-4334-99c5-bc671ae88b01","Type":"ContainerStarted","Data":"c0e30e5ad6d44a8767946a18e63258341266eb3017d47385a2f3edc1e93cb97a"} Sep 15 12:19:54.831935 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.831513 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:19:54.833235 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.833209 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" event={"ID":"9868c351-9407-47ef-a662-fedc60f9d016","Type":"ContainerStarted","Data":"2248a6b3cac123a68f388bdf37b6405c199bb9404b2712c594450e1240ee6ca5"} Sep 15 12:19:54.833365 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.833251 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" event={"ID":"9868c351-9407-47ef-a662-fedc60f9d016","Type":"ContainerStarted","Data":"c610857c986471494ffb5822d44558583165f1a31e22fd8d6a2cf835fabb0c57"} Sep 15 12:19:54.833365 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.833290 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:19:54.854342 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.854298 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" podStartSLOduration=1.8542807190000001 podStartE2EDuration="1.854280719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:19:54.852214551 +0000 UTC m=+1046.609319628" watchObservedRunningTime="2026-09-15 12:19:54.854280719 +0000 UTC m=+1046.611385789" Sep 15 12:19:54.876786 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:19:54.876725 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" podStartSLOduration=1.8767046 podStartE2EDuration="1.8767046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:19:54.875869292 +0000 UTC m=+1046.632974364" watchObservedRunningTime="2026-09-15 12:19:54.8767046 +0000 UTC m=+1046.633809679" Sep 15 12:20:05.842417 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:05.842383 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:20:05.842803 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:05.842461 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:20:13.800695 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:13.800646 2582 patch_prober.go:28] interesting pod/e2e-mt-167943df-openshift-default-974794956-4jxrw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 15 12:20:13.801094 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:13.800722 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 15 12:20:15.965855 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:15.965820 2582 generic.go:356] "Generic (PLEG): container finished" podID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerID="7509c7a894dcd48a797f941e90ad93353d1bccbd2670418d52588d096e46bd77" exitCode=0 Sep 15 12:20:15.966282 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:15.965883 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" event={"ID":"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6","Type":"ContainerDied","Data":"7509c7a894dcd48a797f941e90ad93353d1bccbd2670418d52588d096e46bd77"} Sep 15 12:20:15.966330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:15.966301 2582 scope.go:117] "RemoveContainer" containerID="7509c7a894dcd48a797f941e90ad93353d1bccbd2670418d52588d096e46bd77" Sep 15 12:20:15.967515 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:15.967465 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:20:15.967573 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:15.967555 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:20:15.967609 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:15.967592 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:20:16.818438 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:16.818394 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h"] Sep 15 12:20:16.822950 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:16.822927 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" Sep 15 12:20:16.826823 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:16.826793 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:20:16.826823 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:16.826815 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:20:16.832853 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:16.832827 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h"] Sep 15 12:20:16.890297 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:16.890256 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e70e2045-03f4-4075-bbbf-68cdf40cb608-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-294781-rsq2h\" (UID: \"e70e2045-03f4-4075-bbbf-68cdf40cb608\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" Sep 15 12:20:16.973115 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:16.973080 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" event={"ID":"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6","Type":"ContainerStarted","Data":"46c80eca5bcaf089666f7f09f71b1126678c7b907609bd1e99f0fa46a2ebe700"} Sep 15 12:20:16.990935 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:16.990901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e70e2045-03f4-4075-bbbf-68cdf40cb608-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-294781-rsq2h\" (UID: \"e70e2045-03f4-4075-bbbf-68cdf40cb608\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" Sep 15 12:20:16.991547 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:16.991528 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e70e2045-03f4-4075-bbbf-68cdf40cb608-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-294781-rsq2h\" (UID: \"e70e2045-03f4-4075-bbbf-68cdf40cb608\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" Sep 15 12:20:17.135002 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:17.134907 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" Sep 15 12:20:17.435625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:17.435536 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h"] Sep 15 12:20:17.449420 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:20:17.449383 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70e2045_03f4_4075_bbbf_68cdf40cb608.slice/crio-603d8e8d6e75ea857f316329883c514b55a206f82fe92d17cf7e1a6d191ea1aa WatchSource:0}: Error finding container 603d8e8d6e75ea857f316329883c514b55a206f82fe92d17cf7e1a6d191ea1aa: Status 404 returned error can't find the container with id 603d8e8d6e75ea857f316329883c514b55a206f82fe92d17cf7e1a6d191ea1aa Sep 15 12:20:17.677112 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:17.677072 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:20:17.677471 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:17.677443 2582 patch_prober.go:28] interesting pod/e2e-mt-167943df-openshift-default-974794956-4jxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Sep 15 12:20:17.677600 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:17.677572 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Sep 15 12:20:17.979468 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:17.979433 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" event={"ID":"e70e2045-03f4-4075-bbbf-68cdf40cb608","Type":"ContainerStarted","Data":"909861f7943ea63211815ec527fb27fe01dcb0d87d10e83134eaf244b0263d7c"} Sep 15 12:20:17.979468 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:17.979521 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" event={"ID":"e70e2045-03f4-4075-bbbf-68cdf40cb608","Type":"ContainerStarted","Data":"603d8e8d6e75ea857f316329883c514b55a206f82fe92d17cf7e1a6d191ea1aa"} Sep 15 12:20:18.676761 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:18.676668 2582 patch_prober.go:28] interesting pod/e2e-mt-167943df-openshift-default-974794956-4jxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Sep 15 12:20:18.676761 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:18.676732 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Sep 15 12:20:18.986230 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:18.986198 2582 generic.go:356] "Generic (PLEG): container finished" podID="e70e2045-03f4-4075-bbbf-68cdf40cb608" containerID="909861f7943ea63211815ec527fb27fe01dcb0d87d10e83134eaf244b0263d7c" exitCode=0 Sep 15 12:20:18.986657 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:18.986253 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" event={"ID":"e70e2045-03f4-4075-bbbf-68cdf40cb608","Type":"ContainerDied","Data":"909861f7943ea63211815ec527fb27fe01dcb0d87d10e83134eaf244b0263d7c"} Sep 15 12:20:19.676969 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:19.676932 2582 patch_prober.go:28] interesting pod/e2e-mt-167943df-openshift-default-974794956-4jxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Sep 15 12:20:19.677156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:19.676999 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Sep 15 12:20:20.187273 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:20.187246 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" Sep 15 12:20:20.336270 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:20.336168 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e70e2045-03f4-4075-bbbf-68cdf40cb608-maas-api-service-ca\") pod \"e70e2045-03f4-4075-bbbf-68cdf40cb608\" (UID: \"e70e2045-03f4-4075-bbbf-68cdf40cb608\") " Sep 15 12:20:20.336583 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:20.336557 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e70e2045-03f4-4075-bbbf-68cdf40cb608-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e70e2045-03f4-4075-bbbf-68cdf40cb608" (UID: "e70e2045-03f4-4075-bbbf-68cdf40cb608"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:20:20.437625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:20.437588 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e70e2045-03f4-4075-bbbf-68cdf40cb608-maas-api-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:20.676866 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:20.676829 2582 patch_prober.go:28] interesting pod/e2e-mt-167943df-openshift-default-974794956-4jxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Sep 15 12:20:20.677046 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:20.676888 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Sep 15 12:20:21.000539 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:21.000479 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" Sep 15 12:20:21.000539 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:21.000478 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h" event={"ID":"e70e2045-03f4-4075-bbbf-68cdf40cb608","Type":"ContainerDied","Data":"603d8e8d6e75ea857f316329883c514b55a206f82fe92d17cf7e1a6d191ea1aa"} Sep 15 12:20:21.000820 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:21.000576 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="603d8e8d6e75ea857f316329883c514b55a206f82fe92d17cf7e1a6d191ea1aa" Sep 15 12:20:21.676934 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:21.676898 2582 patch_prober.go:28] interesting pod/e2e-mt-167943df-openshift-default-974794956-4jxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Sep 15 12:20:21.677330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:21.676955 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Sep 15 12:20:22.681778 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:22.681744 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:20:22.682174 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:22.682043 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:20:22.682828 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:22.682812 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:20:25.850467 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:25.850437 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv"] Sep 15 12:20:25.851002 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:25.850984 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70e2045-03f4-4075-bbbf-68cdf40cb608" containerName="revoke-keys" Sep 15 12:20:25.851002 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:25.851003 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70e2045-03f4-4075-bbbf-68cdf40cb608" containerName="revoke-keys" Sep 15 12:20:25.851115 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:25.851070 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70e2045-03f4-4075-bbbf-68cdf40cb608" containerName="revoke-keys" Sep 15 12:20:25.854319 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:25.854299 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" Sep 15 12:20:25.858838 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:25.858804 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:20:25.858944 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:25.858887 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:20:25.870841 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:25.870817 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv"] Sep 15 12:20:25.998152 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:25.998104 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1862161a-c405-4006-a4cd-41b6359e3754-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-167943df-wf8cv\" (UID: \"1862161a-c405-4006-a4cd-41b6359e3754\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" Sep 15 12:20:26.099435 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:26.099392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1862161a-c405-4006-a4cd-41b6359e3754-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-167943df-wf8cv\" (UID: \"1862161a-c405-4006-a4cd-41b6359e3754\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" Sep 15 12:20:26.100456 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:26.100428 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1862161a-c405-4006-a4cd-41b6359e3754-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-167943df-wf8cv\" (UID: \"1862161a-c405-4006-a4cd-41b6359e3754\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" Sep 15 12:20:26.165161 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:26.165072 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" Sep 15 12:20:26.443433 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:26.443405 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv"] Sep 15 12:20:26.445173 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:20:26.445144 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1862161a_c405_4006_a4cd_41b6359e3754.slice/crio-1226a407dc53cb9b8514351ff04cdffc6c217c5965dcd3165c3cfe4ed548cc2a WatchSource:0}: Error finding container 1226a407dc53cb9b8514351ff04cdffc6c217c5965dcd3165c3cfe4ed548cc2a: Status 404 returned error can't find the container with id 1226a407dc53cb9b8514351ff04cdffc6c217c5965dcd3165c3cfe4ed548cc2a Sep 15 12:20:27.043871 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:27.043772 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" event={"ID":"1862161a-c405-4006-a4cd-41b6359e3754","Type":"ContainerStarted","Data":"58b7e3483a85cc482ae39f84fc1e8700d648a910f4d9586b9bfc1e9fcea34a9a"} Sep 15 12:20:27.043871 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:27.043824 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" event={"ID":"1862161a-c405-4006-a4cd-41b6359e3754","Type":"ContainerStarted","Data":"1226a407dc53cb9b8514351ff04cdffc6c217c5965dcd3165c3cfe4ed548cc2a"} Sep 15 12:20:27.073150 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:27.073097 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" podStartSLOduration=2.073083497 podStartE2EDuration="2.073083497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:20:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:20:27.072329031 +0000 UTC m=+1078.829434104" watchObservedRunningTime="2026-09-15 12:20:27.073083497 +0000 UTC m=+1078.830188585" Sep 15 12:20:28.022237 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:28.022198 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h"] Sep 15 12:20:28.037822 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:28.037783 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-294781-rsq2h"] Sep 15 12:20:28.051120 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:28.051086 2582 generic.go:356] "Generic (PLEG): container finished" podID="1862161a-c405-4006-a4cd-41b6359e3754" containerID="58b7e3483a85cc482ae39f84fc1e8700d648a910f4d9586b9bfc1e9fcea34a9a" exitCode=0 Sep 15 12:20:28.051557 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:28.051173 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" event={"ID":"1862161a-c405-4006-a4cd-41b6359e3754","Type":"ContainerDied","Data":"58b7e3483a85cc482ae39f84fc1e8700d648a910f4d9586b9bfc1e9fcea34a9a"} Sep 15 12:20:28.711978 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:28.711951 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e70e2045-03f4-4075-bbbf-68cdf40cb608" path="/var/lib/kubelet/pods/e70e2045-03f4-4075-bbbf-68cdf40cb608/volumes" Sep 15 12:20:29.201834 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:29.201806 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" Sep 15 12:20:29.332840 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:29.332737 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1862161a-c405-4006-a4cd-41b6359e3754-maas-api-service-ca\") pod \"1862161a-c405-4006-a4cd-41b6359e3754\" (UID: \"1862161a-c405-4006-a4cd-41b6359e3754\") " Sep 15 12:20:29.333209 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:29.333181 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1862161a-c405-4006-a4cd-41b6359e3754-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1862161a-c405-4006-a4cd-41b6359e3754" (UID: "1862161a-c405-4006-a4cd-41b6359e3754"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:20:29.434025 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:29.433983 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1862161a-c405-4006-a4cd-41b6359e3754-maas-api-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:30.064170 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:30.064142 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" Sep 15 12:20:30.064170 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:30.064161 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv" event={"ID":"1862161a-c405-4006-a4cd-41b6359e3754","Type":"ContainerDied","Data":"1226a407dc53cb9b8514351ff04cdffc6c217c5965dcd3165c3cfe4ed548cc2a"} Sep 15 12:20:30.064438 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:30.064212 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1226a407dc53cb9b8514351ff04cdffc6c217c5965dcd3165c3cfe4ed548cc2a" Sep 15 12:20:36.914889 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:36.914850 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk"] Sep 15 12:20:36.915373 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:36.915064 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" podUID="4c8dd440-864b-4334-99c5-bc671ae88b01" containerName="payload-processing" containerID="cri-o://3de4b89aa40a01eb5e2d0f84a263fd37ef4068c414020a88857dcbf6346ff267" gracePeriod=30 Sep 15 12:20:36.936075 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:36.936032 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch"] Sep 15 12:20:36.936286 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:36.936262 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" podUID="9868c351-9407-47ef-a662-fedc60f9d016" containerName="payload-pre-processing" containerID="cri-o://2248a6b3cac123a68f388bdf37b6405c199bb9404b2712c594450e1240ee6ca5" gracePeriod=30 Sep 15 12:20:37.126501 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.126445 2582 generic.go:356] "Generic (PLEG): container finished" podID="4c8dd440-864b-4334-99c5-bc671ae88b01" containerID="3de4b89aa40a01eb5e2d0f84a263fd37ef4068c414020a88857dcbf6346ff267" exitCode=0 Sep 15 12:20:37.126681 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.126635 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" event={"ID":"4c8dd440-864b-4334-99c5-bc671ae88b01","Type":"ContainerDied","Data":"3de4b89aa40a01eb5e2d0f84a263fd37ef4068c414020a88857dcbf6346ff267"} Sep 15 12:20:37.128655 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.128630 2582 generic.go:356] "Generic (PLEG): container finished" podID="9868c351-9407-47ef-a662-fedc60f9d016" containerID="2248a6b3cac123a68f388bdf37b6405c199bb9404b2712c594450e1240ee6ca5" exitCode=0 Sep 15 12:20:37.128787 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.128685 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" event={"ID":"9868c351-9407-47ef-a662-fedc60f9d016","Type":"ContainerDied","Data":"2248a6b3cac123a68f388bdf37b6405c199bb9404b2712c594450e1240ee6ca5"} Sep 15 12:20:37.240085 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.239761 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:20:37.245988 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.245961 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:20:37.275832 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.275798 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv"] Sep 15 12:20:37.277921 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.277891 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-167943df-wf8cv"] Sep 15 12:20:37.314414 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.314373 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtv4m\" (UniqueName: \"kubernetes.io/projected/4c8dd440-864b-4334-99c5-bc671ae88b01-kube-api-access-jtv4m\") pod \"4c8dd440-864b-4334-99c5-bc671ae88b01\" (UID: \"4c8dd440-864b-4334-99c5-bc671ae88b01\") " Sep 15 12:20:37.314638 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.314448 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lv9gg\" (UniqueName: \"kubernetes.io/projected/9868c351-9407-47ef-a662-fedc60f9d016-kube-api-access-lv9gg\") pod \"9868c351-9407-47ef-a662-fedc60f9d016\" (UID: \"9868c351-9407-47ef-a662-fedc60f9d016\") " Sep 15 12:20:37.314638 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.314607 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9868c351-9407-47ef-a662-fedc60f9d016-plugins-config-volume\") pod \"9868c351-9407-47ef-a662-fedc60f9d016\" (UID: \"9868c351-9407-47ef-a662-fedc60f9d016\") " Sep 15 12:20:37.314760 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.314664 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c8dd440-864b-4334-99c5-bc671ae88b01-plugins-config-volume\") pod \"4c8dd440-864b-4334-99c5-bc671ae88b01\" (UID: \"4c8dd440-864b-4334-99c5-bc671ae88b01\") " Sep 15 12:20:37.315011 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.314977 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9868c351-9407-47ef-a662-fedc60f9d016-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9868c351-9407-47ef-a662-fedc60f9d016" (UID: "9868c351-9407-47ef-a662-fedc60f9d016"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:20:37.315290 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.315090 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c8dd440-864b-4334-99c5-bc671ae88b01-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4c8dd440-864b-4334-99c5-bc671ae88b01" (UID: "4c8dd440-864b-4334-99c5-bc671ae88b01"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:20:37.315374 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.315353 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9868c351-9407-47ef-a662-fedc60f9d016-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:37.315436 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.315380 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c8dd440-864b-4334-99c5-bc671ae88b01-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:37.316842 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.316811 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c8dd440-864b-4334-99c5-bc671ae88b01-kube-api-access-jtv4m" (OuterVolumeSpecName: "kube-api-access-jtv4m") pod "4c8dd440-864b-4334-99c5-bc671ae88b01" (UID: "4c8dd440-864b-4334-99c5-bc671ae88b01"). InnerVolumeSpecName "kube-api-access-jtv4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:20:37.316939 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.316910 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9868c351-9407-47ef-a662-fedc60f9d016-kube-api-access-lv9gg" (OuterVolumeSpecName: "kube-api-access-lv9gg") pod "9868c351-9407-47ef-a662-fedc60f9d016" (UID: "9868c351-9407-47ef-a662-fedc60f9d016"). InnerVolumeSpecName "kube-api-access-lv9gg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:20:37.416166 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.416130 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtv4m\" (UniqueName: \"kubernetes.io/projected/4c8dd440-864b-4334-99c5-bc671ae88b01-kube-api-access-jtv4m\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:37.416166 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:37.416160 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv9gg\" (UniqueName: \"kubernetes.io/projected/9868c351-9407-47ef-a662-fedc60f9d016-kube-api-access-lv9gg\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:38.135875 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.135841 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" Sep 15 12:20:38.136338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.135847 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk" event={"ID":"4c8dd440-864b-4334-99c5-bc671ae88b01","Type":"ContainerDied","Data":"c0e30e5ad6d44a8767946a18e63258341266eb3017d47385a2f3edc1e93cb97a"} Sep 15 12:20:38.136338 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.135997 2582 scope.go:117] "RemoveContainer" containerID="3de4b89aa40a01eb5e2d0f84a263fd37ef4068c414020a88857dcbf6346ff267" Sep 15 12:20:38.137369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.137343 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" Sep 15 12:20:38.137434 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.137382 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch" event={"ID":"9868c351-9407-47ef-a662-fedc60f9d016","Type":"ContainerDied","Data":"c610857c986471494ffb5822d44558583165f1a31e22fd8d6a2cf835fabb0c57"} Sep 15 12:20:38.148224 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.148016 2582 scope.go:117] "RemoveContainer" containerID="2248a6b3cac123a68f388bdf37b6405c199bb9404b2712c594450e1240ee6ca5" Sep 15 12:20:38.206456 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.206408 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk"] Sep 15 12:20:38.209653 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.209624 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-167943df-5d56fc95fc-gsxjk"] Sep 15 12:20:38.255103 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.255072 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch"] Sep 15 12:20:38.260937 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.260903 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-167943df-7944dbcbf4-8fxch"] Sep 15 12:20:38.712072 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.712033 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1862161a-c405-4006-a4cd-41b6359e3754" path="/var/lib/kubelet/pods/1862161a-c405-4006-a4cd-41b6359e3754/volumes" Sep 15 12:20:38.712578 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.712559 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c8dd440-864b-4334-99c5-bc671ae88b01" path="/var/lib/kubelet/pods/4c8dd440-864b-4334-99c5-bc671ae88b01/volumes" Sep 15 12:20:38.713043 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:38.713027 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9868c351-9407-47ef-a662-fedc60f9d016" path="/var/lib/kubelet/pods/9868c351-9407-47ef-a662-fedc60f9d016/volumes" Sep 15 12:20:45.060688 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:45.060649 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw"] Sep 15 12:20:45.061191 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:45.060893 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" containerID="cri-o://46c80eca5bcaf089666f7f09f71b1126678c7b907609bd1e99f0fa46a2ebe700" gracePeriod=30 Sep 15 12:20:50.211668 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.211636 2582 generic.go:356] "Generic (PLEG): container finished" podID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerID="46c80eca5bcaf089666f7f09f71b1126678c7b907609bd1e99f0fa46a2ebe700" exitCode=0 Sep 15 12:20:50.212108 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.211709 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" event={"ID":"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6","Type":"ContainerDied","Data":"46c80eca5bcaf089666f7f09f71b1126678c7b907609bd1e99f0fa46a2ebe700"} Sep 15 12:20:50.212108 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.211824 2582 scope.go:117] "RemoveContainer" containerID="7509c7a894dcd48a797f941e90ad93353d1bccbd2670418d52588d096e46bd77" Sep 15 12:20:50.324464 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.324403 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:20:50.454393 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454358 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-envoy\") pod \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " Sep 15 12:20:50.454617 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454420 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istiod-ca-cert\") pod \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " Sep 15 12:20:50.454617 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454438 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-data\") pod \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " Sep 15 12:20:50.454617 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454470 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtld6\" (UniqueName: \"kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-kube-api-access-rtld6\") pod \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " Sep 15 12:20:50.454617 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454588 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-podinfo\") pod \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " Sep 15 12:20:50.454857 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454654 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-socket\") pod \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " Sep 15 12:20:50.454857 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454692 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-credential-socket\") pod \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " Sep 15 12:20:50.454857 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454742 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-certs\") pod \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " Sep 15 12:20:50.454857 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454784 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-token\") pod \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\" (UID: \"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6\") " Sep 15 12:20:50.455075 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454935 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" (UID: "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:20:50.455075 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454943 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" (UID: "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:20:50.455075 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.454946 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" (UID: "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:20:50.455242 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.455136 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-data" (OuterVolumeSpecName: "istio-data") pod "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" (UID: "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:20:50.455508 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.455461 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:50.455577 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.455525 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-credential-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:50.455577 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.455542 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istiod-ca-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:50.455577 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.455555 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-data\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:50.455744 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.455612 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" (UID: "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:20:50.456922 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.456898 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-kube-api-access-rtld6" (OuterVolumeSpecName: "kube-api-access-rtld6") pod "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" (UID: "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6"). InnerVolumeSpecName "kube-api-access-rtld6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:20:50.457043 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.456943 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" (UID: "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 12:20:50.457093 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.457042 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" (UID: "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:20:50.457127 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.457102 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-token" (OuterVolumeSpecName: "istio-token") pod "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" (UID: "b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:20:50.556394 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.556359 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-podinfo\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:50.556394 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.556392 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-workload-certs\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:50.556394 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.556403 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-token\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:50.556394 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.556412 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-istio-envoy\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:50.556703 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:50.556421 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtld6\" (UniqueName: \"kubernetes.io/projected/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6-kube-api-access-rtld6\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:20:51.218124 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:51.218093 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" Sep 15 12:20:51.218543 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:51.218129 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw" event={"ID":"b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6","Type":"ContainerDied","Data":"4f7bb285f9cb1b1c07921b7d7420a4391f220657bd3ea41db299cd6e1814f34d"} Sep 15 12:20:51.218543 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:51.218193 2582 scope.go:117] "RemoveContainer" containerID="46c80eca5bcaf089666f7f09f71b1126678c7b907609bd1e99f0fa46a2ebe700" Sep 15 12:20:51.245129 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:51.245096 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw"] Sep 15 12:20:51.248408 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:51.248379 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-167943df-openshift-default-974794956-4jxrw"] Sep 15 12:20:52.712191 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:20:52.712160 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" path="/var/lib/kubelet/pods/b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6/volumes" Sep 15 12:21:10.095554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.095520 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv"] Sep 15 12:21:10.095965 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.095944 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" Sep 15 12:21:10.095965 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.095955 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" Sep 15 12:21:10.096038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.095970 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9868c351-9407-47ef-a662-fedc60f9d016" containerName="payload-pre-processing" Sep 15 12:21:10.096038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.095977 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9868c351-9407-47ef-a662-fedc60f9d016" containerName="payload-pre-processing" Sep 15 12:21:10.096038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.095983 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1862161a-c405-4006-a4cd-41b6359e3754" containerName="revoke-keys" Sep 15 12:21:10.096038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.095989 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1862161a-c405-4006-a4cd-41b6359e3754" containerName="revoke-keys" Sep 15 12:21:10.096038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.095995 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" Sep 15 12:21:10.096038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.096000 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" Sep 15 12:21:10.096038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.096007 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c8dd440-864b-4334-99c5-bc671ae88b01" containerName="payload-processing" Sep 15 12:21:10.096038 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.096012 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c8dd440-864b-4334-99c5-bc671ae88b01" containerName="payload-processing" Sep 15 12:21:10.096277 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.096075 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" Sep 15 12:21:10.096277 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.096086 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b555b01f-ca2b-4bfc-a6e8-90d856ca6bc6" containerName="istio-proxy" Sep 15 12:21:10.096277 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.096099 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1862161a-c405-4006-a4cd-41b6359e3754" containerName="revoke-keys" Sep 15 12:21:10.096277 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.096108 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c8dd440-864b-4334-99c5-bc671ae88b01" containerName="payload-processing" Sep 15 12:21:10.096277 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.096115 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9868c351-9407-47ef-a662-fedc60f9d016" containerName="payload-pre-processing" Sep 15 12:21:10.099406 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.099390 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:21:10.102204 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.102184 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-58b0fd\"" Sep 15 12:21:10.114110 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.114088 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv"] Sep 15 12:21:10.122012 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.121987 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n"] Sep 15 12:21:10.125507 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.125476 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:21:10.129140 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.129115 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-58b0fd-dockercfg-zk2fk\"" Sep 15 12:21:10.143138 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.143095 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n"] Sep 15 12:21:10.153588 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.153553 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2kfd\" (UniqueName: \"kubernetes.io/projected/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-kube-api-access-b2kfd\") pod \"payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n\" (UID: \"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:21:10.153727 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.153613 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2230fc2b-d18c-4acc-8f0a-60a5138c125b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv\" (UID: \"2230fc2b-d18c-4acc-8f0a-60a5138c125b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:21:10.153727 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.153681 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n\" (UID: \"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:21:10.153821 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.153755 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7ddp\" (UniqueName: \"kubernetes.io/projected/2230fc2b-d18c-4acc-8f0a-60a5138c125b-kube-api-access-s7ddp\") pod \"payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv\" (UID: \"2230fc2b-d18c-4acc-8f0a-60a5138c125b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:21:10.257243 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.256167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n\" (UID: \"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:21:10.257542 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.257215 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n\" (UID: \"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:21:10.257749 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.257727 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7ddp\" (UniqueName: \"kubernetes.io/projected/2230fc2b-d18c-4acc-8f0a-60a5138c125b-kube-api-access-s7ddp\") pod \"payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv\" (UID: \"2230fc2b-d18c-4acc-8f0a-60a5138c125b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:21:10.258195 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.258171 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2kfd\" (UniqueName: \"kubernetes.io/projected/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-kube-api-access-b2kfd\") pod \"payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n\" (UID: \"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:21:10.258363 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.258348 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2230fc2b-d18c-4acc-8f0a-60a5138c125b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv\" (UID: \"2230fc2b-d18c-4acc-8f0a-60a5138c125b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:21:10.259278 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.259246 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2230fc2b-d18c-4acc-8f0a-60a5138c125b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv\" (UID: \"2230fc2b-d18c-4acc-8f0a-60a5138c125b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:21:10.271689 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.271660 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7ddp\" (UniqueName: \"kubernetes.io/projected/2230fc2b-d18c-4acc-8f0a-60a5138c125b-kube-api-access-s7ddp\") pod \"payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv\" (UID: \"2230fc2b-d18c-4acc-8f0a-60a5138c125b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:21:10.271923 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.271904 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2kfd\" (UniqueName: \"kubernetes.io/projected/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-kube-api-access-b2kfd\") pod \"payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n\" (UID: \"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:21:10.410094 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.410004 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:21:10.436830 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.436792 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:21:10.695791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.695754 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv"] Sep 15 12:21:10.696814 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:21:10.696792 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2230fc2b_d18c_4acc_8f0a_60a5138c125b.slice/crio-ebc1f779a5d87abf3704f80c38f09ce81f4654c31198ec13ab806124783843ee WatchSource:0}: Error finding container ebc1f779a5d87abf3704f80c38f09ce81f4654c31198ec13ab806124783843ee: Status 404 returned error can't find the container with id ebc1f779a5d87abf3704f80c38f09ce81f4654c31198ec13ab806124783843ee Sep 15 12:21:10.731918 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:10.731896 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n"] Sep 15 12:21:10.732787 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:21:10.732758 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b92dfd_ad27_4eb7_9cc3_2166dd4cb306.slice/crio-d4a90da591105cec316671a24c23a170eadcb9471b509602980368b767f5a1ee WatchSource:0}: Error finding container d4a90da591105cec316671a24c23a170eadcb9471b509602980368b767f5a1ee: Status 404 returned error can't find the container with id d4a90da591105cec316671a24c23a170eadcb9471b509602980368b767f5a1ee Sep 15 12:21:11.329567 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:11.329527 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" event={"ID":"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306","Type":"ContainerStarted","Data":"bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629"} Sep 15 12:21:11.330045 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:11.329589 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" event={"ID":"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306","Type":"ContainerStarted","Data":"d4a90da591105cec316671a24c23a170eadcb9471b509602980368b767f5a1ee"} Sep 15 12:21:11.330045 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:11.329616 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:21:11.331322 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:11.331296 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" event={"ID":"2230fc2b-d18c-4acc-8f0a-60a5138c125b","Type":"ContainerStarted","Data":"30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039"} Sep 15 12:21:11.331451 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:11.331338 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" event={"ID":"2230fc2b-d18c-4acc-8f0a-60a5138c125b","Type":"ContainerStarted","Data":"ebc1f779a5d87abf3704f80c38f09ce81f4654c31198ec13ab806124783843ee"} Sep 15 12:21:11.331451 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:11.331386 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:21:11.354364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:11.354312 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" podStartSLOduration=1.35429601 podStartE2EDuration="1.35429601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:21:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:21:11.352316696 +0000 UTC m=+1123.109421768" watchObservedRunningTime="2026-09-15 12:21:11.35429601 +0000 UTC m=+1123.111401129" Sep 15 12:21:11.371589 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:11.371539 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" podStartSLOduration=1.371520473 podStartE2EDuration="1.371520473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:21:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:21:11.370514231 +0000 UTC m=+1123.127619302" watchObservedRunningTime="2026-09-15 12:21:11.371520473 +0000 UTC m=+1123.128625546" Sep 15 12:21:22.339924 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:22.339891 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:21:22.340380 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:22.340053 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:21:35.480056 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.480017 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh"] Sep 15 12:21:35.485348 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.485327 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.488367 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.488344 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-ba366c-openshift-default-dockercfg-b22rw\"" Sep 15 12:21:35.519351 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.519308 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh"] Sep 15 12:21:35.525847 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.525816 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-certs\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.525998 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.525855 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-credential-socket\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.525998 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.525912 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-data\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.525998 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.525931 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-podinfo\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.525998 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.525972 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-socket\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.526150 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.526038 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-envoy\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.526150 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.526084 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgqhv\" (UniqueName: \"kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-kube-api-access-hgqhv\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.526150 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.526103 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istiod-ca-cert\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.526150 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.526142 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-token\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.627357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.627320 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-certs\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.627954 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.627922 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-certs\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.628467 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.628437 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-credential-socket\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.628752 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.628732 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-data\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.628885 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.628871 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-podinfo\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.628991 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.628977 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-socket\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.629104 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.629088 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-envoy\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.629289 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.629271 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgqhv\" (UniqueName: \"kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-kube-api-access-hgqhv\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.629418 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.629403 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istiod-ca-cert\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.629554 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.629539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-token\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.630551 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.630527 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-socket\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.632283 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.632255 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-credential-socket\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.632381 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.632269 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-data\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.633261 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.633237 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istiod-ca-cert\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.634007 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.633983 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-envoy\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.634451 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.634426 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-podinfo\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.640282 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.640251 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-token\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.643578 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.643552 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgqhv\" (UniqueName: \"kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-kube-api-access-hgqhv\") pod \"e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:35.796863 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:35.796771 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:36.086802 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:36.086708 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh"] Sep 15 12:21:36.089261 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:21:36.089234 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c5aff6b_cd5e_4092_b33f_9574119b8d82.slice/crio-b49ce3455d7f66d9f9d47a69c46fab0a7b0e5e38f365ba1e39f27a3ab568674f WatchSource:0}: Error finding container b49ce3455d7f66d9f9d47a69c46fab0a7b0e5e38f365ba1e39f27a3ab568674f: Status 404 returned error can't find the container with id b49ce3455d7f66d9f9d47a69c46fab0a7b0e5e38f365ba1e39f27a3ab568674f Sep 15 12:21:36.091787 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:36.091740 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:21:36.091882 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:36.091845 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:21:36.091925 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:36.091891 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:21:36.468668 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:36.468628 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" event={"ID":"0c5aff6b-cd5e-4092-b33f-9574119b8d82","Type":"ContainerStarted","Data":"7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a"} Sep 15 12:21:36.468856 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:36.468702 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" event={"ID":"0c5aff6b-cd5e-4092-b33f-9574119b8d82","Type":"ContainerStarted","Data":"b49ce3455d7f66d9f9d47a69c46fab0a7b0e5e38f365ba1e39f27a3ab568674f"} Sep 15 12:21:36.506412 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:36.506358 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" podStartSLOduration=1.5063417989999999 podStartE2EDuration="1.506341799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:21:36.504642717 +0000 UTC m=+1148.261747787" watchObservedRunningTime="2026-09-15 12:21:36.506341799 +0000 UTC m=+1148.263446869" Sep 15 12:21:36.796980 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:36.796889 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:37.803305 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:37.803270 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:38.480390 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:38.480356 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:38.481471 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:38.481445 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:21:44.764164 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.764121 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq"] Sep 15 12:21:44.769808 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.769779 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:21:44.773826 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.773797 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-ba366c-dockercfg-bbzzf\"" Sep 15 12:21:44.774581 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.774558 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-ba366c\"" Sep 15 12:21:44.776165 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.776136 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq"] Sep 15 12:21:44.831879 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.831843 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d0c377e-10d0-491c-9825-93311a12a414-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq\" (UID: \"2d0c377e-10d0-491c-9825-93311a12a414\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:21:44.832064 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.831931 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbwp5\" (UniqueName: \"kubernetes.io/projected/2d0c377e-10d0-491c-9825-93311a12a414-kube-api-access-tbwp5\") pod \"payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq\" (UID: \"2d0c377e-10d0-491c-9825-93311a12a414\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:21:44.933495 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.933459 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbwp5\" (UniqueName: \"kubernetes.io/projected/2d0c377e-10d0-491c-9825-93311a12a414-kube-api-access-tbwp5\") pod \"payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq\" (UID: \"2d0c377e-10d0-491c-9825-93311a12a414\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:21:44.933698 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.933604 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d0c377e-10d0-491c-9825-93311a12a414-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq\" (UID: \"2d0c377e-10d0-491c-9825-93311a12a414\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:21:44.934282 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.934263 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d0c377e-10d0-491c-9825-93311a12a414-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq\" (UID: \"2d0c377e-10d0-491c-9825-93311a12a414\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:21:44.943616 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:44.943590 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbwp5\" (UniqueName: \"kubernetes.io/projected/2d0c377e-10d0-491c-9825-93311a12a414-kube-api-access-tbwp5\") pod \"payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq\" (UID: \"2d0c377e-10d0-491c-9825-93311a12a414\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:21:45.083280 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:45.083192 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:21:45.370432 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:45.370398 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq"] Sep 15 12:21:45.371649 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:21:45.371619 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d0c377e_10d0_491c_9825_93311a12a414.slice/crio-6e5f9c2e260b5512acbdd3729c4c4a5bc281fe68bb9e58ebe34986e97254b808 WatchSource:0}: Error finding container 6e5f9c2e260b5512acbdd3729c4c4a5bc281fe68bb9e58ebe34986e97254b808: Status 404 returned error can't find the container with id 6e5f9c2e260b5512acbdd3729c4c4a5bc281fe68bb9e58ebe34986e97254b808 Sep 15 12:21:45.520581 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:45.520547 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" event={"ID":"2d0c377e-10d0-491c-9825-93311a12a414","Type":"ContainerStarted","Data":"6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c"} Sep 15 12:21:45.520760 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:45.520598 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" event={"ID":"2d0c377e-10d0-491c-9825-93311a12a414","Type":"ContainerStarted","Data":"6e5f9c2e260b5512acbdd3729c4c4a5bc281fe68bb9e58ebe34986e97254b808"} Sep 15 12:21:45.520807 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:45.520759 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:21:45.551889 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:45.551839 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" podStartSLOduration=1.55182116 podStartE2EDuration="1.55182116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:21:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:21:45.551309906 +0000 UTC m=+1157.308414978" watchObservedRunningTime="2026-09-15 12:21:45.55182116 +0000 UTC m=+1157.308926271" Sep 15 12:21:56.529659 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:21:56.529622 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:22:04.310568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:04.310529 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb"] Sep 15 12:22:04.316854 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:04.316819 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" Sep 15 12:22:04.319904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:04.319877 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:22:04.320354 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:04.320333 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:22:04.331947 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:04.331916 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb"] Sep 15 12:22:04.348065 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:04.348029 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f62122-b5c8-494e-a212-5b5257acffe8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb\" (UID: \"45f62122-b5c8-494e-a212-5b5257acffe8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" Sep 15 12:22:04.449592 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:04.449554 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f62122-b5c8-494e-a212-5b5257acffe8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb\" (UID: \"45f62122-b5c8-494e-a212-5b5257acffe8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" Sep 15 12:22:04.450169 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:04.450149 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f62122-b5c8-494e-a212-5b5257acffe8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb\" (UID: \"45f62122-b5c8-494e-a212-5b5257acffe8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" Sep 15 12:22:04.634717 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:04.634630 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" Sep 15 12:22:04.914684 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:04.914657 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb"] Sep 15 12:22:04.916906 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:22:04.916878 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45f62122_b5c8_494e_a212_5b5257acffe8.slice/crio-d01d8f52f5d199ed536f903972ffe8a4974ae6801d5e7847ba19e624ca662242 WatchSource:0}: Error finding container d01d8f52f5d199ed536f903972ffe8a4974ae6801d5e7847ba19e624ca662242: Status 404 returned error can't find the container with id d01d8f52f5d199ed536f903972ffe8a4974ae6801d5e7847ba19e624ca662242 Sep 15 12:22:05.634647 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:05.634614 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" event={"ID":"45f62122-b5c8-494e-a212-5b5257acffe8","Type":"ContainerStarted","Data":"0fbc1734a84712a9a9fc7c46501d4ea94f40c7b8588619c741d687fa7cce84ca"} Sep 15 12:22:05.635108 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:05.634664 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" event={"ID":"45f62122-b5c8-494e-a212-5b5257acffe8","Type":"ContainerStarted","Data":"d01d8f52f5d199ed536f903972ffe8a4974ae6801d5e7847ba19e624ca662242"} Sep 15 12:22:05.655402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:05.655341 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" podStartSLOduration=1.655318921 podStartE2EDuration="1.655318921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:22:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:22:05.654373759 +0000 UTC m=+1177.411478833" watchObservedRunningTime="2026-09-15 12:22:05.655318921 +0000 UTC m=+1177.412423993" Sep 15 12:22:06.642228 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:06.642191 2582 generic.go:356] "Generic (PLEG): container finished" podID="45f62122-b5c8-494e-a212-5b5257acffe8" containerID="0fbc1734a84712a9a9fc7c46501d4ea94f40c7b8588619c741d687fa7cce84ca" exitCode=0 Sep 15 12:22:06.642671 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:06.642280 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" event={"ID":"45f62122-b5c8-494e-a212-5b5257acffe8","Type":"ContainerDied","Data":"0fbc1734a84712a9a9fc7c46501d4ea94f40c7b8588619c741d687fa7cce84ca"} Sep 15 12:22:07.828291 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:07.828189 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" Sep 15 12:22:07.887648 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:07.887621 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f62122-b5c8-494e-a212-5b5257acffe8-maas-api-service-ca\") pod \"45f62122-b5c8-494e-a212-5b5257acffe8\" (UID: \"45f62122-b5c8-494e-a212-5b5257acffe8\") " Sep 15 12:22:07.887992 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:07.887972 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45f62122-b5c8-494e-a212-5b5257acffe8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "45f62122-b5c8-494e-a212-5b5257acffe8" (UID: "45f62122-b5c8-494e-a212-5b5257acffe8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:22:07.989158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:07.989122 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f62122-b5c8-494e-a212-5b5257acffe8-maas-api-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:08.655396 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:08.655368 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" Sep 15 12:22:08.655588 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:08.655368 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb" event={"ID":"45f62122-b5c8-494e-a212-5b5257acffe8","Type":"ContainerDied","Data":"d01d8f52f5d199ed536f903972ffe8a4974ae6801d5e7847ba19e624ca662242"} Sep 15 12:22:08.655588 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:08.655517 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d01d8f52f5d199ed536f903972ffe8a4974ae6801d5e7847ba19e624ca662242" Sep 15 12:22:14.691251 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:14.691216 2582 generic.go:356] "Generic (PLEG): container finished" podID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerID="7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a" exitCode=0 Sep 15 12:22:14.691694 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:14.691290 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" event={"ID":"0c5aff6b-cd5e-4092-b33f-9574119b8d82","Type":"ContainerDied","Data":"7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a"} Sep 15 12:22:14.691744 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:14.691698 2582 scope.go:117] "RemoveContainer" containerID="7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a" Sep 15 12:22:14.692825 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:14.692798 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:22:14.692893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:14.692864 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:22:14.692930 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:14.692896 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:22:15.699244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:15.699206 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" event={"ID":"0c5aff6b-cd5e-4092-b33f-9574119b8d82","Type":"ContainerStarted","Data":"b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d"} Sep 15 12:22:15.797237 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:15.797205 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:22:15.797451 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:15.797428 2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 15 12:22:15.797577 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:15.797504 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 15 12:22:16.458339 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.458307 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k"] Sep 15 12:22:16.458737 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.458578 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" podUID="04ab8ba0-ae23-4442-b8cc-5d02311a4228" containerName="payload-pre-processing" containerID="cri-o://90168908313bafdac9203501703298335a3f61b8bb633851fe3158c6049c4234" gracePeriod=30 Sep 15 12:22:16.732877 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.730301 2582 generic.go:356] "Generic (PLEG): container finished" podID="04ab8ba0-ae23-4442-b8cc-5d02311a4228" containerID="90168908313bafdac9203501703298335a3f61b8bb633851fe3158c6049c4234" exitCode=0 Sep 15 12:22:16.737370 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.737315 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" event={"ID":"04ab8ba0-ae23-4442-b8cc-5d02311a4228","Type":"ContainerDied","Data":"90168908313bafdac9203501703298335a3f61b8bb633851fe3158c6049c4234"} Sep 15 12:22:16.784457 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.784422 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb"] Sep 15 12:22:16.790586 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.790558 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:22:16.791527 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.791503 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9453-5jbdb"] Sep 15 12:22:16.797398 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.797367 2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 15 12:22:16.797543 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.797422 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 15 12:22:16.888275 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.888240 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7sml\" (UniqueName: \"kubernetes.io/projected/04ab8ba0-ae23-4442-b8cc-5d02311a4228-kube-api-access-c7sml\") pod \"04ab8ba0-ae23-4442-b8cc-5d02311a4228\" (UID: \"04ab8ba0-ae23-4442-b8cc-5d02311a4228\") " Sep 15 12:22:16.888275 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.888282 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04ab8ba0-ae23-4442-b8cc-5d02311a4228-plugins-config-volume\") pod \"04ab8ba0-ae23-4442-b8cc-5d02311a4228\" (UID: \"04ab8ba0-ae23-4442-b8cc-5d02311a4228\") " Sep 15 12:22:16.888657 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.888635 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04ab8ba0-ae23-4442-b8cc-5d02311a4228-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "04ab8ba0-ae23-4442-b8cc-5d02311a4228" (UID: "04ab8ba0-ae23-4442-b8cc-5d02311a4228"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:22:16.890610 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.890587 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ab8ba0-ae23-4442-b8cc-5d02311a4228-kube-api-access-c7sml" (OuterVolumeSpecName: "kube-api-access-c7sml") pod "04ab8ba0-ae23-4442-b8cc-5d02311a4228" (UID: "04ab8ba0-ae23-4442-b8cc-5d02311a4228"). InnerVolumeSpecName "kube-api-access-c7sml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:22:16.989320 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.989229 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7sml\" (UniqueName: \"kubernetes.io/projected/04ab8ba0-ae23-4442-b8cc-5d02311a4228-kube-api-access-c7sml\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:16.989320 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:16.989263 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04ab8ba0-ae23-4442-b8cc-5d02311a4228-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:17.738560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:17.738529 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" Sep 15 12:22:17.738560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:17.738552 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k" event={"ID":"04ab8ba0-ae23-4442-b8cc-5d02311a4228","Type":"ContainerDied","Data":"a081cbb74cc20db60d2d0eb754f1d41adaa12e579b6600fee8c40bfd02514148"} Sep 15 12:22:17.739138 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:17.738626 2582 scope.go:117] "RemoveContainer" containerID="90168908313bafdac9203501703298335a3f61b8bb633851fe3158c6049c4234" Sep 15 12:22:17.762712 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:17.762678 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k"] Sep 15 12:22:17.764695 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:17.764662 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4a9453-54b9fc874b-pjr5k"] Sep 15 12:22:17.797432 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:17.797398 2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 15 12:22:17.797647 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:17.797452 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 15 12:22:18.721906 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:18.721873 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04ab8ba0-ae23-4442-b8cc-5d02311a4228" path="/var/lib/kubelet/pods/04ab8ba0-ae23-4442-b8cc-5d02311a4228/volumes" Sep 15 12:22:18.722776 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:18.722754 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45f62122-b5c8-494e-a212-5b5257acffe8" path="/var/lib/kubelet/pods/45f62122-b5c8-494e-a212-5b5257acffe8/volumes" Sep 15 12:22:18.797833 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:18.797794 2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 15 12:22:18.798236 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:18.797853 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 15 12:22:19.797544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:19.797461 2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 15 12:22:19.797698 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:19.797564 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 15 12:22:20.798061 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:20.798021 2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Sep 15 12:22:20.798560 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:20.798090 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Sep 15 12:22:21.801547 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:21.801507 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:22:21.802016 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:21.801792 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:22:21.802533 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:21.802517 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:22:25.572991 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:25.572957 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5"] Sep 15 12:22:25.573391 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:25.573166 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" containerID="cri-o://206c5dc76f2d60bc3267978c49e69d52eaff6df1c2c2068c4d4517737547a734" gracePeriod=30 Sep 15 12:22:30.814330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.814303 2582 generic.go:356] "Generic (PLEG): container finished" podID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerID="206c5dc76f2d60bc3267978c49e69d52eaff6df1c2c2068c4d4517737547a734" exitCode=0 Sep 15 12:22:30.814688 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.814354 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" event={"ID":"59fd9969-fdeb-4a2b-988e-650fb17da2cc","Type":"ContainerDied","Data":"206c5dc76f2d60bc3267978c49e69d52eaff6df1c2c2068c4d4517737547a734"} Sep 15 12:22:30.814688 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.814404 2582 scope.go:117] "RemoveContainer" containerID="2acf7032cfd45a8a66538bbc598999270346b557cdf3d3da882b2daf094eec28" Sep 15 12:22:30.837166 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:22:30.837143 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2acf7032cfd45a8a66538bbc598999270346b557cdf3d3da882b2daf094eec28\": container with ID starting with 2acf7032cfd45a8a66538bbc598999270346b557cdf3d3da882b2daf094eec28 not found: ID does not exist" containerID="2acf7032cfd45a8a66538bbc598999270346b557cdf3d3da882b2daf094eec28" Sep 15 12:22:30.841008 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.840985 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:22:30.844197 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844177 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-credential-socket\") pod \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " Sep 15 12:22:30.844330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844221 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-token\") pod \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " Sep 15 12:22:30.844330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844269 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-data\") pod \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " Sep 15 12:22:30.844330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844285 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-socket\") pod \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " Sep 15 12:22:30.844330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844321 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-podinfo\") pod \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " Sep 15 12:22:30.844606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844375 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwg9h\" (UniqueName: \"kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-kube-api-access-lwg9h\") pod \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " Sep 15 12:22:30.844606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844382 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59fd9969-fdeb-4a2b-988e-650fb17da2cc" (UID: "59fd9969-fdeb-4a2b-988e-650fb17da2cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:22:30.844606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844402 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istiod-ca-cert\") pod \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " Sep 15 12:22:30.844606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844436 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-certs\") pod \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " Sep 15 12:22:30.844606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844496 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-envoy\") pod \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\" (UID: \"59fd9969-fdeb-4a2b-988e-650fb17da2cc\") " Sep 15 12:22:30.844606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844568 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "59fd9969-fdeb-4a2b-988e-650fb17da2cc" (UID: "59fd9969-fdeb-4a2b-988e-650fb17da2cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:22:30.844925 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844754 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59fd9969-fdeb-4a2b-988e-650fb17da2cc" (UID: "59fd9969-fdeb-4a2b-988e-650fb17da2cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:22:30.844925 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844839 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59fd9969-fdeb-4a2b-988e-650fb17da2cc" (UID: "59fd9969-fdeb-4a2b-988e-650fb17da2cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:22:30.845050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844967 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59fd9969-fdeb-4a2b-988e-650fb17da2cc" (UID: "59fd9969-fdeb-4a2b-988e-650fb17da2cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:22:30.845050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.844995 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-credential-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:30.845050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.845014 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-data\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:30.845050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.845027 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istiod-ca-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:30.845050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.845038 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-certs\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:30.847012 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.846982 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59fd9969-fdeb-4a2b-988e-650fb17da2cc" (UID: "59fd9969-fdeb-4a2b-988e-650fb17da2cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 12:22:30.847107 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.847024 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "59fd9969-fdeb-4a2b-988e-650fb17da2cc" (UID: "59fd9969-fdeb-4a2b-988e-650fb17da2cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:22:30.847171 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.847142 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59fd9969-fdeb-4a2b-988e-650fb17da2cc" (UID: "59fd9969-fdeb-4a2b-988e-650fb17da2cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:22:30.847300 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.847279 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-kube-api-access-lwg9h" (OuterVolumeSpecName: "kube-api-access-lwg9h") pod "59fd9969-fdeb-4a2b-988e-650fb17da2cc" (UID: "59fd9969-fdeb-4a2b-988e-650fb17da2cc"). InnerVolumeSpecName "kube-api-access-lwg9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:22:30.946349 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.946314 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-token\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:30.946349 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.946345 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-workload-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:30.946349 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.946356 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-podinfo\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:30.946620 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.946364 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwg9h\" (UniqueName: \"kubernetes.io/projected/59fd9969-fdeb-4a2b-988e-650fb17da2cc-kube-api-access-lwg9h\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:30.946620 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:30.946373 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59fd9969-fdeb-4a2b-988e-650fb17da2cc-istio-envoy\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:31.821851 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:31.821816 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" event={"ID":"59fd9969-fdeb-4a2b-988e-650fb17da2cc","Type":"ContainerDied","Data":"231a8ba910a0285cd456bf777380ea953ada3bab0f688ab018bc0d9b6b70c306"} Sep 15 12:22:31.822285 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:31.821871 2582 scope.go:117] "RemoveContainer" containerID="206c5dc76f2d60bc3267978c49e69d52eaff6df1c2c2068c4d4517737547a734" Sep 15 12:22:31.822285 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:31.821830 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5" Sep 15 12:22:31.853662 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:31.853624 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5"] Sep 15 12:22:31.862787 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:31.862749 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4a9453-openshift-default-7457bbfb9d-k2bw5"] Sep 15 12:22:32.166833 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.166747 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc"] Sep 15 12:22:32.167202 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167190 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" Sep 15 12:22:32.167247 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167203 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" Sep 15 12:22:32.167247 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167213 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" Sep 15 12:22:32.167247 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167218 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" Sep 15 12:22:32.167247 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167234 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ab8ba0-ae23-4442-b8cc-5d02311a4228" containerName="payload-pre-processing" Sep 15 12:22:32.167247 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167239 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ab8ba0-ae23-4442-b8cc-5d02311a4228" containerName="payload-pre-processing" Sep 15 12:22:32.167397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167250 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45f62122-b5c8-494e-a212-5b5257acffe8" containerName="revoke-keys" Sep 15 12:22:32.167397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167255 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="45f62122-b5c8-494e-a212-5b5257acffe8" containerName="revoke-keys" Sep 15 12:22:32.167397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167314 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="45f62122-b5c8-494e-a212-5b5257acffe8" containerName="revoke-keys" Sep 15 12:22:32.167397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167323 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="04ab8ba0-ae23-4442-b8cc-5d02311a4228" containerName="payload-pre-processing" Sep 15 12:22:32.167397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.167330 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" Sep 15 12:22:32.172343 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.172324 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:22:32.175359 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.175335 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-80b72486\"" Sep 15 12:22:32.175516 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.175362 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-80b72486-dockercfg-kxkh4\"" Sep 15 12:22:32.184534 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.184513 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc"] Sep 15 12:22:32.257011 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.256972 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc\" (UID: \"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:22:32.257011 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.257013 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27krg\" (UniqueName: \"kubernetes.io/projected/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-kube-api-access-27krg\") pod \"payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc\" (UID: \"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:22:32.357916 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.357879 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc\" (UID: \"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:22:32.358102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.357927 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27krg\" (UniqueName: \"kubernetes.io/projected/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-kube-api-access-27krg\") pod \"payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc\" (UID: \"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:22:32.358685 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.358663 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc\" (UID: \"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:22:32.367550 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.367521 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27krg\" (UniqueName: \"kubernetes.io/projected/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-kube-api-access-27krg\") pod \"payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc\" (UID: \"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:22:32.484259 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.484221 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:22:32.712589 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.712557 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" path="/var/lib/kubelet/pods/59fd9969-fdeb-4a2b-988e-650fb17da2cc/volumes" Sep 15 12:22:32.767102 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:22:32.767076 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad1cc76_1225_4de1_a246_fbd1cd81ebdd.slice/crio-3e391bb838cf766fb3d703dc19b1074bf12ed4b2bf29c8aa6d53deb7908ab178 WatchSource:0}: Error finding container 3e391bb838cf766fb3d703dc19b1074bf12ed4b2bf29c8aa6d53deb7908ab178: Status 404 returned error can't find the container with id 3e391bb838cf766fb3d703dc19b1074bf12ed4b2bf29c8aa6d53deb7908ab178 Sep 15 12:22:32.768231 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.768204 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc"] Sep 15 12:22:32.828760 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:32.828727 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" event={"ID":"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd","Type":"ContainerStarted","Data":"3e391bb838cf766fb3d703dc19b1074bf12ed4b2bf29c8aa6d53deb7908ab178"} Sep 15 12:22:33.836818 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:33.836778 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" event={"ID":"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd","Type":"ContainerStarted","Data":"9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791"} Sep 15 12:22:33.837219 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:33.836970 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:22:33.863646 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:33.863593 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" podStartSLOduration=1.863575327 podStartE2EDuration="1.863575327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:22:33.862653992 +0000 UTC m=+1205.619759085" watchObservedRunningTime="2026-09-15 12:22:33.863575327 +0000 UTC m=+1205.620680398" Sep 15 12:22:36.819580 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:36.819548 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p"] Sep 15 12:22:36.820028 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:36.819756 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" podUID="8e017f15-2533-4da1-83ce-0fc49d8c832f" containerName="payload-processing" containerID="cri-o://99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee" gracePeriod=30 Sep 15 12:22:37.084977 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.084893 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:22:37.109190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.108949 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e017f15-2533-4da1-83ce-0fc49d8c832f-plugins-config-volume\") pod \"8e017f15-2533-4da1-83ce-0fc49d8c832f\" (UID: \"8e017f15-2533-4da1-83ce-0fc49d8c832f\") " Sep 15 12:22:37.109190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.109069 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq8pg\" (UniqueName: \"kubernetes.io/projected/8e017f15-2533-4da1-83ce-0fc49d8c832f-kube-api-access-qq8pg\") pod \"8e017f15-2533-4da1-83ce-0fc49d8c832f\" (UID: \"8e017f15-2533-4da1-83ce-0fc49d8c832f\") " Sep 15 12:22:37.109455 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.109404 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e017f15-2533-4da1-83ce-0fc49d8c832f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e017f15-2533-4da1-83ce-0fc49d8c832f" (UID: "8e017f15-2533-4da1-83ce-0fc49d8c832f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:22:37.111693 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.111662 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e017f15-2533-4da1-83ce-0fc49d8c832f-kube-api-access-qq8pg" (OuterVolumeSpecName: "kube-api-access-qq8pg") pod "8e017f15-2533-4da1-83ce-0fc49d8c832f" (UID: "8e017f15-2533-4da1-83ce-0fc49d8c832f"). InnerVolumeSpecName "kube-api-access-qq8pg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:22:37.210444 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.210408 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e017f15-2533-4da1-83ce-0fc49d8c832f-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:37.210444 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.210446 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq8pg\" (UniqueName: \"kubernetes.io/projected/8e017f15-2533-4da1-83ce-0fc49d8c832f-kube-api-access-qq8pg\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:22:37.864804 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.864769 2582 generic.go:356] "Generic (PLEG): container finished" podID="8e017f15-2533-4da1-83ce-0fc49d8c832f" containerID="99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee" exitCode=0 Sep 15 12:22:37.865278 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.864817 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" Sep 15 12:22:37.865278 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.864816 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" event={"ID":"8e017f15-2533-4da1-83ce-0fc49d8c832f","Type":"ContainerDied","Data":"99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee"} Sep 15 12:22:37.865278 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.864941 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p" event={"ID":"8e017f15-2533-4da1-83ce-0fc49d8c832f","Type":"ContainerDied","Data":"55c42c715f9eb54399f8560abb5b3e991545521d28fca588c25d27343aa28e2f"} Sep 15 12:22:37.865278 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.864963 2582 scope.go:117] "RemoveContainer" containerID="99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee" Sep 15 12:22:37.874320 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.874304 2582 scope.go:117] "RemoveContainer" containerID="99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee" Sep 15 12:22:37.874629 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:22:37.874610 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee\": container with ID starting with 99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee not found: ID does not exist" containerID="99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee" Sep 15 12:22:37.874697 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.874638 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee"} err="failed to get container status \"99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee\": rpc error: code = NotFound desc = could not find container \"99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee\": container with ID starting with 99e87644152bc273af27addf0c77c018db8e38a2b9b0b0c232b41610923459ee not found: ID does not exist" Sep 15 12:22:37.898870 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.898844 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p"] Sep 15 12:22:37.908035 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:37.908008 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-69d7ba-5d4f87d7b9-5982p"] Sep 15 12:22:38.712343 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:38.712308 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e017f15-2533-4da1-83ce-0fc49d8c832f" path="/var/lib/kubelet/pods/8e017f15-2533-4da1-83ce-0fc49d8c832f/volumes" Sep 15 12:22:44.844661 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:44.844627 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:22:55.894499 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:55.894462 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc"] Sep 15 12:22:55.894956 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:55.894911 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e017f15-2533-4da1-83ce-0fc49d8c832f" containerName="payload-processing" Sep 15 12:22:55.894956 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:55.894923 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e017f15-2533-4da1-83ce-0fc49d8c832f" containerName="payload-processing" Sep 15 12:22:55.895072 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:55.895002 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e017f15-2533-4da1-83ce-0fc49d8c832f" containerName="payload-processing" Sep 15 12:22:55.895072 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:55.895011 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="59fd9969-fdeb-4a2b-988e-650fb17da2cc" containerName="istio-proxy" Sep 15 12:22:55.897954 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:55.897936 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:22:55.906431 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:55.906406 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-1dc430-dockercfg-rm9bk\"" Sep 15 12:22:55.906595 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:55.906399 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-1dc430\"" Sep 15 12:22:55.916070 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:55.916043 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc"] Sep 15 12:22:56.008709 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.008678 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n52f\" (UniqueName: \"kubernetes.io/projected/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-kube-api-access-9n52f\") pod \"payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc\" (UID: \"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:22:56.008872 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.008740 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc\" (UID: \"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:22:56.110270 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.110224 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n52f\" (UniqueName: \"kubernetes.io/projected/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-kube-api-access-9n52f\") pod \"payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc\" (UID: \"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:22:56.110460 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.110296 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc\" (UID: \"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:22:56.111037 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.111015 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc\" (UID: \"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:22:56.119414 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.119389 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n52f\" (UniqueName: \"kubernetes.io/projected/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-kube-api-access-9n52f\") pod \"payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc\" (UID: \"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:22:56.208856 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.208763 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:22:56.494298 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.494269 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc"] Sep 15 12:22:56.496377 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:22:56.496343 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ddee2bb_0d48_41c7_ac7d_3bf8420f4f11.slice/crio-54e7106d95717a29f97af65fb996fe8d93f8dac96bd934d4538b39a6b2c19760 WatchSource:0}: Error finding container 54e7106d95717a29f97af65fb996fe8d93f8dac96bd934d4538b39a6b2c19760: Status 404 returned error can't find the container with id 54e7106d95717a29f97af65fb996fe8d93f8dac96bd934d4538b39a6b2c19760 Sep 15 12:22:56.992925 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.992889 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" event={"ID":"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11","Type":"ContainerStarted","Data":"da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e"} Sep 15 12:22:56.993333 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.992940 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" event={"ID":"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11","Type":"ContainerStarted","Data":"54e7106d95717a29f97af65fb996fe8d93f8dac96bd934d4538b39a6b2c19760"} Sep 15 12:22:56.993333 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:56.993037 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:22:57.015192 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:22:57.015137 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" podStartSLOduration=2.015120686 podStartE2EDuration="2.015120686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:22:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:22:57.012834643 +0000 UTC m=+1228.769939714" watchObservedRunningTime="2026-09-15 12:22:57.015120686 +0000 UTC m=+1228.772225757" Sep 15 12:23:08.001347 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:08.001317 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:23:26.387606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.387566 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p"] Sep 15 12:23:26.391196 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.391175 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:23:26.394622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.394602 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-969130\"" Sep 15 12:23:26.408042 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.408016 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p"] Sep 15 12:23:26.413019 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.412993 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx"] Sep 15 12:23:26.416714 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.416692 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:23:26.419466 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:23:26.419432 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-e2e-shared-b-w3-969130-dockercfg-zh8jj\" is forbidden: User \"system:node:ip-10-0-130-139.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-139.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-969130-dockercfg-zh8jj\"" type="*v1.Secret" Sep 15 12:23:26.448946 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.448910 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx"] Sep 15 12:23:26.526941 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.525091 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4858017-7123-467f-98dc-ecfa71491749-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx\" (UID: \"d4858017-7123-467f-98dc-ecfa71491749\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:23:26.526941 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.525147 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khsxh\" (UniqueName: \"kubernetes.io/projected/d4858017-7123-467f-98dc-ecfa71491749-kube-api-access-khsxh\") pod \"payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx\" (UID: \"d4858017-7123-467f-98dc-ecfa71491749\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:23:26.526941 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.525209 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p\" (UID: \"e2468531-ddb4-40ca-9d3a-9c884fe24d8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:23:26.526941 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.525233 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkkl7\" (UniqueName: \"kubernetes.io/projected/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-kube-api-access-xkkl7\") pod \"payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p\" (UID: \"e2468531-ddb4-40ca-9d3a-9c884fe24d8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:23:26.626050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.626013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4858017-7123-467f-98dc-ecfa71491749-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx\" (UID: \"d4858017-7123-467f-98dc-ecfa71491749\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:23:26.626050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.626055 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khsxh\" (UniqueName: \"kubernetes.io/projected/d4858017-7123-467f-98dc-ecfa71491749-kube-api-access-khsxh\") pod \"payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx\" (UID: \"d4858017-7123-467f-98dc-ecfa71491749\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:23:26.626264 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.626095 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p\" (UID: \"e2468531-ddb4-40ca-9d3a-9c884fe24d8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:23:26.626264 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.626114 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkkl7\" (UniqueName: \"kubernetes.io/projected/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-kube-api-access-xkkl7\") pod \"payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p\" (UID: \"e2468531-ddb4-40ca-9d3a-9c884fe24d8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:23:26.626896 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.626873 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4858017-7123-467f-98dc-ecfa71491749-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx\" (UID: \"d4858017-7123-467f-98dc-ecfa71491749\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:23:26.626945 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.626897 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p\" (UID: \"e2468531-ddb4-40ca-9d3a-9c884fe24d8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:23:26.654416 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.654336 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkkl7\" (UniqueName: \"kubernetes.io/projected/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-kube-api-access-xkkl7\") pod \"payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p\" (UID: \"e2468531-ddb4-40ca-9d3a-9c884fe24d8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:23:26.659912 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.659886 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khsxh\" (UniqueName: \"kubernetes.io/projected/d4858017-7123-467f-98dc-ecfa71491749-kube-api-access-khsxh\") pod \"payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx\" (UID: \"d4858017-7123-467f-98dc-ecfa71491749\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:23:26.701910 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.701847 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:23:26.995334 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:26.995299 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p"] Sep 15 12:23:26.996139 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:23:26.996113 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2468531_ddb4_40ca_9d3a_9c884fe24d8a.slice/crio-023d11163acb97bae2131d5b74e464e09574d56a00632af245a578e4c28567b2 WatchSource:0}: Error finding container 023d11163acb97bae2131d5b74e464e09574d56a00632af245a578e4c28567b2: Status 404 returned error can't find the container with id 023d11163acb97bae2131d5b74e464e09574d56a00632af245a578e4c28567b2 Sep 15 12:23:27.158802 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:27.158757 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" event={"ID":"e2468531-ddb4-40ca-9d3a-9c884fe24d8a","Type":"ContainerStarted","Data":"b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d"} Sep 15 12:23:27.158972 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:27.158815 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" event={"ID":"e2468531-ddb4-40ca-9d3a-9c884fe24d8a","Type":"ContainerStarted","Data":"023d11163acb97bae2131d5b74e464e09574d56a00632af245a578e4c28567b2"} Sep 15 12:23:27.158972 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:27.158840 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:23:27.183158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:27.183107 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" podStartSLOduration=1.183091372 podStartE2EDuration="1.183091372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:23:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:23:27.181677502 +0000 UTC m=+1258.938782573" watchObservedRunningTime="2026-09-15 12:23:27.183091372 +0000 UTC m=+1258.940196442" Sep 15 12:23:27.449967 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:27.449876 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-969130-dockercfg-zh8jj\"" Sep 15 12:23:27.457720 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:27.457695 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:23:27.738518 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:27.738465 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx"] Sep 15 12:23:27.739369 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:23:27.739343 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4858017_7123_467f_98dc_ecfa71491749.slice/crio-c1e3adc287589b0a0aaba326c66d1c6d2fab0bbd4281965efc372a1906eba651 WatchSource:0}: Error finding container c1e3adc287589b0a0aaba326c66d1c6d2fab0bbd4281965efc372a1906eba651: Status 404 returned error can't find the container with id c1e3adc287589b0a0aaba326c66d1c6d2fab0bbd4281965efc372a1906eba651 Sep 15 12:23:28.166424 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:28.166335 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" event={"ID":"d4858017-7123-467f-98dc-ecfa71491749","Type":"ContainerStarted","Data":"341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327"} Sep 15 12:23:28.166424 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:28.166394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" event={"ID":"d4858017-7123-467f-98dc-ecfa71491749","Type":"ContainerStarted","Data":"c1e3adc287589b0a0aaba326c66d1c6d2fab0bbd4281965efc372a1906eba651"} Sep 15 12:23:28.201548 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:28.201480 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" podStartSLOduration=2.20146486 podStartE2EDuration="2.20146486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:23:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:23:28.199246319 +0000 UTC m=+1259.956351395" watchObservedRunningTime="2026-09-15 12:23:28.20146486 +0000 UTC m=+1259.958569932" Sep 15 12:23:28.815204 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:28.815173 2582 scope.go:117] "RemoveContainer" containerID="7d2b78868ba65c84c4340f491e25162d4304403a4eca5ed26e926331e496b6bf" Sep 15 12:23:29.174557 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:29.174460 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:23:38.168473 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:38.168438 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:23:40.182287 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:23:40.182253 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:24:12.158682 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:12.158643 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws"] Sep 15 12:24:12.162622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:12.162604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws" Sep 15 12:24:12.166651 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:12.166632 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:24:12.166803 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:12.166633 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:24:12.171115 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:12.171092 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws"] Sep 15 12:24:12.320050 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:12.320011 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a69ee14-fdee-4f30-8d65-0286b7c65ac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-80b72486-4f5ws\" (UID: \"1a69ee14-fdee-4f30-8d65-0286b7c65ac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws" Sep 15 12:24:12.420970 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:12.420881 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a69ee14-fdee-4f30-8d65-0286b7c65ac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-80b72486-4f5ws\" (UID: \"1a69ee14-fdee-4f30-8d65-0286b7c65ac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws" Sep 15 12:24:12.421515 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:12.421480 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a69ee14-fdee-4f30-8d65-0286b7c65ac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-80b72486-4f5ws\" (UID: \"1a69ee14-fdee-4f30-8d65-0286b7c65ac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws" Sep 15 12:24:12.475741 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:12.475700 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws" Sep 15 12:24:12.753899 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:12.753870 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws"] Sep 15 12:24:12.756056 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:24:12.756027 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a69ee14_fdee_4f30_8d65_0286b7c65ac4.slice/crio-383a12d255953ecb8075a5459f6261a3772ff1fd45de8d65276d6097484647c9 WatchSource:0}: Error finding container 383a12d255953ecb8075a5459f6261a3772ff1fd45de8d65276d6097484647c9: Status 404 returned error can't find the container with id 383a12d255953ecb8075a5459f6261a3772ff1fd45de8d65276d6097484647c9 Sep 15 12:24:13.428212 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:13.428180 2582 generic.go:356] "Generic (PLEG): container finished" podID="1a69ee14-fdee-4f30-8d65-0286b7c65ac4" containerID="70453d617970433943125696ad20f6dceff1aaf3a5f491a3541eecf5cac06e19" exitCode=0 Sep 15 12:24:13.428553 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:13.428233 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws" event={"ID":"1a69ee14-fdee-4f30-8d65-0286b7c65ac4","Type":"ContainerDied","Data":"70453d617970433943125696ad20f6dceff1aaf3a5f491a3541eecf5cac06e19"} Sep 15 12:24:13.428553 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:13.428280 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws" event={"ID":"1a69ee14-fdee-4f30-8d65-0286b7c65ac4","Type":"ContainerStarted","Data":"383a12d255953ecb8075a5459f6261a3772ff1fd45de8d65276d6097484647c9"} Sep 15 12:24:14.589975 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:14.589946 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws" Sep 15 12:24:14.642822 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:14.642792 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a69ee14-fdee-4f30-8d65-0286b7c65ac4-maas-api-service-ca\") pod \"1a69ee14-fdee-4f30-8d65-0286b7c65ac4\" (UID: \"1a69ee14-fdee-4f30-8d65-0286b7c65ac4\") " Sep 15 12:24:14.643155 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:14.643131 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a69ee14-fdee-4f30-8d65-0286b7c65ac4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1a69ee14-fdee-4f30-8d65-0286b7c65ac4" (UID: "1a69ee14-fdee-4f30-8d65-0286b7c65ac4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:24:14.744153 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:14.744111 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a69ee14-fdee-4f30-8d65-0286b7c65ac4-maas-api-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:24:15.443567 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:15.443535 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws" Sep 15 12:24:15.443749 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:15.443535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws" event={"ID":"1a69ee14-fdee-4f30-8d65-0286b7c65ac4","Type":"ContainerDied","Data":"383a12d255953ecb8075a5459f6261a3772ff1fd45de8d65276d6097484647c9"} Sep 15 12:24:15.443749 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:15.443661 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="383a12d255953ecb8075a5459f6261a3772ff1fd45de8d65276d6097484647c9" Sep 15 12:24:27.949646 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:27.949610 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc"] Sep 15 12:24:27.950082 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:27.949863 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" podUID="6ad1cc76-1225-4de1-a246-fbd1cd81ebdd" containerName="payload-pre-processing" containerID="cri-o://9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791" gracePeriod=30 Sep 15 12:24:28.173081 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.173044 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws"] Sep 15 12:24:28.178406 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.178373 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-80b72486-4f5ws"] Sep 15 12:24:28.239533 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.239472 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:24:28.277965 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.277917 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27krg\" (UniqueName: \"kubernetes.io/projected/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-kube-api-access-27krg\") pod \"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd\" (UID: \"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd\") " Sep 15 12:24:28.278157 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.278010 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-plugins-config-volume\") pod \"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd\" (UID: \"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd\") " Sep 15 12:24:28.279257 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.278792 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ad1cc76-1225-4de1-a246-fbd1cd81ebdd" (UID: "6ad1cc76-1225-4de1-a246-fbd1cd81ebdd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:24:28.281403 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.281357 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-kube-api-access-27krg" (OuterVolumeSpecName: "kube-api-access-27krg") pod "6ad1cc76-1225-4de1-a246-fbd1cd81ebdd" (UID: "6ad1cc76-1225-4de1-a246-fbd1cd81ebdd"). InnerVolumeSpecName "kube-api-access-27krg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:24:28.379273 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.379232 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27krg\" (UniqueName: \"kubernetes.io/projected/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-kube-api-access-27krg\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:24:28.379273 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.379266 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:24:28.519416 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.519325 2582 generic.go:356] "Generic (PLEG): container finished" podID="6ad1cc76-1225-4de1-a246-fbd1cd81ebdd" containerID="9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791" exitCode=0 Sep 15 12:24:28.519416 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.519373 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" Sep 15 12:24:28.519654 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.519422 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" event={"ID":"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd","Type":"ContainerDied","Data":"9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791"} Sep 15 12:24:28.519654 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.519523 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc" event={"ID":"6ad1cc76-1225-4de1-a246-fbd1cd81ebdd","Type":"ContainerDied","Data":"3e391bb838cf766fb3d703dc19b1074bf12ed4b2bf29c8aa6d53deb7908ab178"} Sep 15 12:24:28.519654 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.519553 2582 scope.go:117] "RemoveContainer" containerID="9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791" Sep 15 12:24:28.529364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.529345 2582 scope.go:117] "RemoveContainer" containerID="9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791" Sep 15 12:24:28.529641 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:24:28.529622 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791\": container with ID starting with 9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791 not found: ID does not exist" containerID="9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791" Sep 15 12:24:28.529709 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.529649 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791"} err="failed to get container status \"9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791\": rpc error: code = NotFound desc = could not find container \"9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791\": container with ID starting with 9d786956448b8250ee91c01c4eeb6e7520966cadbce6e867caf9e7b7210cd791 not found: ID does not exist" Sep 15 12:24:28.543820 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.543789 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc"] Sep 15 12:24:28.547065 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.547045 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-80b72486-7658c877f8-qnkxc"] Sep 15 12:24:28.712246 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.712211 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a69ee14-fdee-4f30-8d65-0286b7c65ac4" path="/var/lib/kubelet/pods/1a69ee14-fdee-4f30-8d65-0286b7c65ac4/volumes" Sep 15 12:24:28.712608 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:24:28.712595 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ad1cc76-1225-4de1-a246-fbd1cd81ebdd" path="/var/lib/kubelet/pods/6ad1cc76-1225-4de1-a246-fbd1cd81ebdd/volumes" Sep 15 12:26:26.720425 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.720396 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg"] Sep 15 12:26:26.720946 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.720930 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad1cc76-1225-4de1-a246-fbd1cd81ebdd" containerName="payload-pre-processing" Sep 15 12:26:26.720946 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.720948 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad1cc76-1225-4de1-a246-fbd1cd81ebdd" containerName="payload-pre-processing" Sep 15 12:26:26.721025 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.720957 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a69ee14-fdee-4f30-8d65-0286b7c65ac4" containerName="revoke-keys" Sep 15 12:26:26.721025 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.720963 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a69ee14-fdee-4f30-8d65-0286b7c65ac4" containerName="revoke-keys" Sep 15 12:26:26.721091 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.721033 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a69ee14-fdee-4f30-8d65-0286b7c65ac4" containerName="revoke-keys" Sep 15 12:26:26.721091 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.721043 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad1cc76-1225-4de1-a246-fbd1cd81ebdd" containerName="payload-pre-processing" Sep 15 12:26:26.724543 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.724523 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:26:26.728662 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.728633 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-f1e118\"" Sep 15 12:26:26.728813 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.728792 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-f1e118-dockercfg-qbfdv\"" Sep 15 12:26:26.742935 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.742911 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km89l\" (UniqueName: \"kubernetes.io/projected/59a90509-f186-4483-899b-c3812a2e2c42-kube-api-access-km89l\") pod \"payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg\" (UID: \"59a90509-f186-4483-899b-c3812a2e2c42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:26:26.743066 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.742953 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59a90509-f186-4483-899b-c3812a2e2c42-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg\" (UID: \"59a90509-f186-4483-899b-c3812a2e2c42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:26:26.753271 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.753242 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg"] Sep 15 12:26:26.824277 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.824238 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct"] Sep 15 12:26:26.829910 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.829883 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:26:26.844438 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.844407 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff42a837-75b8-457c-83d4-035aa491314e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct\" (UID: \"ff42a837-75b8-457c-83d4-035aa491314e\") " pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:26:26.844623 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.844501 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs5qg\" (UniqueName: \"kubernetes.io/projected/ff42a837-75b8-457c-83d4-035aa491314e-kube-api-access-gs5qg\") pod \"payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct\" (UID: \"ff42a837-75b8-457c-83d4-035aa491314e\") " pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:26:26.844623 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.844553 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km89l\" (UniqueName: \"kubernetes.io/projected/59a90509-f186-4483-899b-c3812a2e2c42-kube-api-access-km89l\") pod \"payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg\" (UID: \"59a90509-f186-4483-899b-c3812a2e2c42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:26:26.844623 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.844596 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59a90509-f186-4483-899b-c3812a2e2c42-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg\" (UID: \"59a90509-f186-4483-899b-c3812a2e2c42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:26:26.844826 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.844805 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct"] Sep 15 12:26:26.845365 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.845343 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59a90509-f186-4483-899b-c3812a2e2c42-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg\" (UID: \"59a90509-f186-4483-899b-c3812a2e2c42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:26:26.858122 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.858095 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km89l\" (UniqueName: \"kubernetes.io/projected/59a90509-f186-4483-899b-c3812a2e2c42-kube-api-access-km89l\") pod \"payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg\" (UID: \"59a90509-f186-4483-899b-c3812a2e2c42\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:26:26.945243 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.945205 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs5qg\" (UniqueName: \"kubernetes.io/projected/ff42a837-75b8-457c-83d4-035aa491314e-kube-api-access-gs5qg\") pod \"payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct\" (UID: \"ff42a837-75b8-457c-83d4-035aa491314e\") " pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:26:26.945453 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.945317 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff42a837-75b8-457c-83d4-035aa491314e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct\" (UID: \"ff42a837-75b8-457c-83d4-035aa491314e\") " pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:26:26.945974 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.945954 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff42a837-75b8-457c-83d4-035aa491314e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct\" (UID: \"ff42a837-75b8-457c-83d4-035aa491314e\") " pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:26:26.954807 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:26.954778 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs5qg\" (UniqueName: \"kubernetes.io/projected/ff42a837-75b8-457c-83d4-035aa491314e-kube-api-access-gs5qg\") pod \"payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct\" (UID: \"ff42a837-75b8-457c-83d4-035aa491314e\") " pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:26:27.035313 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:27.035173 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:26:27.140091 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:27.140060 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:26:27.322062 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:27.322040 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg"] Sep 15 12:26:27.323983 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:26:27.323958 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a90509_f186_4483_899b_c3812a2e2c42.slice/crio-1ada4ebff5b8a467e58de5daec01f583375a957423a5837e09830a5e100c3c2b WatchSource:0}: Error finding container 1ada4ebff5b8a467e58de5daec01f583375a957423a5837e09830a5e100c3c2b: Status 404 returned error can't find the container with id 1ada4ebff5b8a467e58de5daec01f583375a957423a5837e09830a5e100c3c2b Sep 15 12:26:27.424225 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:27.424200 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct"] Sep 15 12:26:27.426286 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:26:27.426258 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff42a837_75b8_457c_83d4_035aa491314e.slice/crio-8657624cd5e1e714c33f3b4c3aa704793ee05b67d681f12f10c32a982550de4f WatchSource:0}: Error finding container 8657624cd5e1e714c33f3b4c3aa704793ee05b67d681f12f10c32a982550de4f: Status 404 returned error can't find the container with id 8657624cd5e1e714c33f3b4c3aa704793ee05b67d681f12f10c32a982550de4f Sep 15 12:26:28.207211 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:28.207169 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" event={"ID":"ff42a837-75b8-457c-83d4-035aa491314e","Type":"ContainerStarted","Data":"a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3"} Sep 15 12:26:28.207724 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:28.207235 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" event={"ID":"ff42a837-75b8-457c-83d4-035aa491314e","Type":"ContainerStarted","Data":"8657624cd5e1e714c33f3b4c3aa704793ee05b67d681f12f10c32a982550de4f"} Sep 15 12:26:28.207724 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:28.207365 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:26:28.208876 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:28.208856 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" event={"ID":"59a90509-f186-4483-899b-c3812a2e2c42","Type":"ContainerStarted","Data":"c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a"} Sep 15 12:26:28.208956 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:28.208893 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" event={"ID":"59a90509-f186-4483-899b-c3812a2e2c42","Type":"ContainerStarted","Data":"1ada4ebff5b8a467e58de5daec01f583375a957423a5837e09830a5e100c3c2b"} Sep 15 12:26:28.209013 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:28.208951 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:26:28.290871 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:28.290814 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" podStartSLOduration=2.29079503 podStartE2EDuration="2.29079503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:28.290768041 +0000 UTC m=+1440.047873111" watchObservedRunningTime="2026-09-15 12:26:28.29079503 +0000 UTC m=+1440.047900092" Sep 15 12:26:28.291245 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:28.291222 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" podStartSLOduration=2.291217015 podStartE2EDuration="2.291217015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:26:28.247401243 +0000 UTC m=+1440.004506317" watchObservedRunningTime="2026-09-15 12:26:28.291217015 +0000 UTC m=+1440.048322143" Sep 15 12:26:28.873568 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:28.873523 2582 scope.go:117] "RemoveContainer" containerID="58b7e3483a85cc482ae39f84fc1e8700d648a910f4d9586b9bfc1e9fcea34a9a" Sep 15 12:26:28.882200 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:28.882181 2582 scope.go:117] "RemoveContainer" containerID="909861f7943ea63211815ec527fb27fe01dcb0d87d10e83134eaf244b0263d7c" Sep 15 12:26:39.217673 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:39.217641 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:26:39.218132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:26:39.217822 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:27:08.203698 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.203664 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr"] Sep 15 12:27:08.207436 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.207416 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:27:08.210930 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.210909 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-eb9bbe-dockercfg-llvrf\"" Sep 15 12:27:08.211071 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.211051 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-eb9bbe\"" Sep 15 12:27:08.223358 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.223325 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr"] Sep 15 12:27:08.281734 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.281697 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-plugins-config-volume\") pod \"payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr\" (UID: \"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931\") " pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:27:08.281734 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.281733 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ql28\" (UniqueName: \"kubernetes.io/projected/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-kube-api-access-8ql28\") pod \"payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr\" (UID: \"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931\") " pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:27:08.383258 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.383215 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-plugins-config-volume\") pod \"payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr\" (UID: \"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931\") " pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:27:08.383258 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.383260 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ql28\" (UniqueName: \"kubernetes.io/projected/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-kube-api-access-8ql28\") pod \"payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr\" (UID: \"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931\") " pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:27:08.383941 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.383922 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-plugins-config-volume\") pod \"payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr\" (UID: \"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931\") " pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:27:08.392684 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.392660 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ql28\" (UniqueName: \"kubernetes.io/projected/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-kube-api-access-8ql28\") pod \"payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr\" (UID: \"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931\") " pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:27:08.518105 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.518068 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:27:08.802878 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:08.802790 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr"] Sep 15 12:27:08.805955 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:27:08.805928 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aecbcd6_b459_419b_8a7e_ccc9ccb7b931.slice/crio-26a514db3ee08362bd3ca1d110611899545b70287643b1b0bb383dc2d4f3edb5 WatchSource:0}: Error finding container 26a514db3ee08362bd3ca1d110611899545b70287643b1b0bb383dc2d4f3edb5: Status 404 returned error can't find the container with id 26a514db3ee08362bd3ca1d110611899545b70287643b1b0bb383dc2d4f3edb5 Sep 15 12:27:09.446741 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:09.446709 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" event={"ID":"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931","Type":"ContainerStarted","Data":"06a2483c05f7f577fcebab08f53099ef507ca144ed42b57e3fcce5f65117f5ee"} Sep 15 12:27:09.447171 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:09.446767 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" event={"ID":"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931","Type":"ContainerStarted","Data":"26a514db3ee08362bd3ca1d110611899545b70287643b1b0bb383dc2d4f3edb5"} Sep 15 12:27:09.447171 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:09.446846 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:27:09.476714 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:09.476659 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" podStartSLOduration=1.476634908 podStartE2EDuration="1.476634908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:27:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:27:09.475560142 +0000 UTC m=+1481.232665213" watchObservedRunningTime="2026-09-15 12:27:09.476634908 +0000 UTC m=+1481.233740024" Sep 15 12:27:17.357525 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.357473 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn"] Sep 15 12:27:17.360132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.358203 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" podUID="b3444551-04fb-482e-9f2f-5a74395afc19" containerName="payload-processing" containerID="cri-o://6afe172570d1ba53821f41a0fa87a9cba435c227972c1fad23a9aa28f28d8785" gracePeriod=30 Sep 15 12:27:17.362754 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.362316 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj"] Sep 15 12:27:17.363378 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.363326 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" podUID="b8ad94c8-d45e-435b-aa5b-ead9d00f802e" containerName="payload-pre-processing" containerID="cri-o://41e18a24ff906e3105cfaab2191e032484b72b4a3394d01a2cec66811e4ad65d" gracePeriod=30 Sep 15 12:27:17.495549 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.495514 2582 generic.go:356] "Generic (PLEG): container finished" podID="b8ad94c8-d45e-435b-aa5b-ead9d00f802e" containerID="41e18a24ff906e3105cfaab2191e032484b72b4a3394d01a2cec66811e4ad65d" exitCode=0 Sep 15 12:27:17.495767 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.495588 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" event={"ID":"b8ad94c8-d45e-435b-aa5b-ead9d00f802e","Type":"ContainerDied","Data":"41e18a24ff906e3105cfaab2191e032484b72b4a3394d01a2cec66811e4ad65d"} Sep 15 12:27:17.497736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.497708 2582 generic.go:356] "Generic (PLEG): container finished" podID="b3444551-04fb-482e-9f2f-5a74395afc19" containerID="6afe172570d1ba53821f41a0fa87a9cba435c227972c1fad23a9aa28f28d8785" exitCode=0 Sep 15 12:27:17.497864 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.497749 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" event={"ID":"b3444551-04fb-482e-9f2f-5a74395afc19","Type":"ContainerDied","Data":"6afe172570d1ba53821f41a0fa87a9cba435c227972c1fad23a9aa28f28d8785"} Sep 15 12:27:17.672044 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.672010 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:27:17.674206 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.674185 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:27:17.785957 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.785921 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pjkn\" (UniqueName: \"kubernetes.io/projected/b3444551-04fb-482e-9f2f-5a74395afc19-kube-api-access-4pjkn\") pod \"b3444551-04fb-482e-9f2f-5a74395afc19\" (UID: \"b3444551-04fb-482e-9f2f-5a74395afc19\") " Sep 15 12:27:17.786158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.786008 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-plugins-config-volume\") pod \"b8ad94c8-d45e-435b-aa5b-ead9d00f802e\" (UID: \"b8ad94c8-d45e-435b-aa5b-ead9d00f802e\") " Sep 15 12:27:17.786158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.786086 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3444551-04fb-482e-9f2f-5a74395afc19-plugins-config-volume\") pod \"b3444551-04fb-482e-9f2f-5a74395afc19\" (UID: \"b3444551-04fb-482e-9f2f-5a74395afc19\") " Sep 15 12:27:17.786158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.786115 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppwbl\" (UniqueName: \"kubernetes.io/projected/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-kube-api-access-ppwbl\") pod \"b8ad94c8-d45e-435b-aa5b-ead9d00f802e\" (UID: \"b8ad94c8-d45e-435b-aa5b-ead9d00f802e\") " Sep 15 12:27:17.789618 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.786975 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3444551-04fb-482e-9f2f-5a74395afc19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b3444551-04fb-482e-9f2f-5a74395afc19" (UID: "b3444551-04fb-482e-9f2f-5a74395afc19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:27:17.789618 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.787382 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8ad94c8-d45e-435b-aa5b-ead9d00f802e" (UID: "b8ad94c8-d45e-435b-aa5b-ead9d00f802e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:27:17.789618 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.789282 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3444551-04fb-482e-9f2f-5a74395afc19-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:17.789618 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.789303 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:17.791735 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.791696 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3444551-04fb-482e-9f2f-5a74395afc19-kube-api-access-4pjkn" (OuterVolumeSpecName: "kube-api-access-4pjkn") pod "b3444551-04fb-482e-9f2f-5a74395afc19" (UID: "b3444551-04fb-482e-9f2f-5a74395afc19"). InnerVolumeSpecName "kube-api-access-4pjkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:27:17.791986 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.791950 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-kube-api-access-ppwbl" (OuterVolumeSpecName: "kube-api-access-ppwbl") pod "b8ad94c8-d45e-435b-aa5b-ead9d00f802e" (UID: "b8ad94c8-d45e-435b-aa5b-ead9d00f802e"). InnerVolumeSpecName "kube-api-access-ppwbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:27:17.890462 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.890370 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pjkn\" (UniqueName: \"kubernetes.io/projected/b3444551-04fb-482e-9f2f-5a74395afc19-kube-api-access-4pjkn\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:17.890462 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:17.890406 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppwbl\" (UniqueName: \"kubernetes.io/projected/b8ad94c8-d45e-435b-aa5b-ead9d00f802e-kube-api-access-ppwbl\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:18.504184 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.504152 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" Sep 15 12:27:18.504184 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.504172 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn" event={"ID":"b3444551-04fb-482e-9f2f-5a74395afc19","Type":"ContainerDied","Data":"72a381142b21898f3da9f0f5c5e502ee4ac33cc0e77f0fa439332ef375d1c63a"} Sep 15 12:27:18.504762 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.504240 2582 scope.go:117] "RemoveContainer" containerID="6afe172570d1ba53821f41a0fa87a9cba435c227972c1fad23a9aa28f28d8785" Sep 15 12:27:18.505783 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.505760 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" Sep 15 12:27:18.505859 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.505796 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj" event={"ID":"b8ad94c8-d45e-435b-aa5b-ead9d00f802e","Type":"ContainerDied","Data":"edc480c2c5ecede848e0ad232c24a1b922c58382807acc65a7774ba046709d85"} Sep 15 12:27:18.525054 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.524782 2582 scope.go:117] "RemoveContainer" containerID="41e18a24ff906e3105cfaab2191e032484b72b4a3394d01a2cec66811e4ad65d" Sep 15 12:27:18.546986 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.546951 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj"] Sep 15 12:27:18.550933 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.550904 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-680229-6c7745fdc4-t6mvj"] Sep 15 12:27:18.589428 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.589395 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn"] Sep 15 12:27:18.602275 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.602244 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-680229-7d588558d5-8jdmn"] Sep 15 12:27:18.712378 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.712344 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3444551-04fb-482e-9f2f-5a74395afc19" path="/var/lib/kubelet/pods/b3444551-04fb-482e-9f2f-5a74395afc19/volumes" Sep 15 12:27:18.712777 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:18.712761 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8ad94c8-d45e-435b-aa5b-ead9d00f802e" path="/var/lib/kubelet/pods/b8ad94c8-d45e-435b-aa5b-ead9d00f802e/volumes" Sep 15 12:27:20.454589 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:20.454559 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:27:26.654056 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:26.654019 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6"] Sep 15 12:27:26.654566 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:26.654260 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" containerID="cri-o://14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd" gracePeriod=30 Sep 15 12:27:31.920621 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.920556 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:27:31.940659 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.940626 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-istio-token\") pod \"2736624f-181c-4b3b-9088-3b08bde5c003\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " Sep 15 12:27:31.940837 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.940671 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-envoy\") pod \"2736624f-181c-4b3b-9088-3b08bde5c003\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " Sep 15 12:27:31.940837 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.940696 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9l5g\" (UniqueName: \"kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-kube-api-access-t9l5g\") pod \"2736624f-181c-4b3b-9088-3b08bde5c003\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " Sep 15 12:27:31.940837 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.940808 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-data\") pod \"2736624f-181c-4b3b-9088-3b08bde5c003\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " Sep 15 12:27:31.941007 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.940873 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-socket\") pod \"2736624f-181c-4b3b-9088-3b08bde5c003\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " Sep 15 12:27:31.941007 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.940925 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-certs\") pod \"2736624f-181c-4b3b-9088-3b08bde5c003\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " Sep 15 12:27:31.941007 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.940992 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2736624f-181c-4b3b-9088-3b08bde5c003-istio-podinfo\") pod \"2736624f-181c-4b3b-9088-3b08bde5c003\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " Sep 15 12:27:31.941146 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941039 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-data" (OuterVolumeSpecName: "istio-data") pod "2736624f-181c-4b3b-9088-3b08bde5c003" (UID: "2736624f-181c-4b3b-9088-3b08bde5c003"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:27:31.941146 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941072 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-credential-socket\") pod \"2736624f-181c-4b3b-9088-3b08bde5c003\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " Sep 15 12:27:31.941260 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941143 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2736624f-181c-4b3b-9088-3b08bde5c003-istiod-ca-cert\") pod \"2736624f-181c-4b3b-9088-3b08bde5c003\" (UID: \"2736624f-181c-4b3b-9088-3b08bde5c003\") " Sep 15 12:27:31.941260 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941147 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2736624f-181c-4b3b-9088-3b08bde5c003" (UID: "2736624f-181c-4b3b-9088-3b08bde5c003"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:27:31.941260 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941235 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2736624f-181c-4b3b-9088-3b08bde5c003" (UID: "2736624f-181c-4b3b-9088-3b08bde5c003"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:27:31.941569 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941546 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2736624f-181c-4b3b-9088-3b08bde5c003" (UID: "2736624f-181c-4b3b-9088-3b08bde5c003"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:27:31.941653 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941573 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2736624f-181c-4b3b-9088-3b08bde5c003-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2736624f-181c-4b3b-9088-3b08bde5c003" (UID: "2736624f-181c-4b3b-9088-3b08bde5c003"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:27:31.941777 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941755 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:31.942002 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941784 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-workload-certs\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:31.942002 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941800 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-credential-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:31.942002 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941815 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2736624f-181c-4b3b-9088-3b08bde5c003-istiod-ca-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:31.942002 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.941829 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-data\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:31.943057 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.943032 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2736624f-181c-4b3b-9088-3b08bde5c003" (UID: "2736624f-181c-4b3b-9088-3b08bde5c003"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:27:31.943158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.943055 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-istio-token" (OuterVolumeSpecName: "istio-token") pod "2736624f-181c-4b3b-9088-3b08bde5c003" (UID: "2736624f-181c-4b3b-9088-3b08bde5c003"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:27:31.943158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.943084 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-kube-api-access-t9l5g" (OuterVolumeSpecName: "kube-api-access-t9l5g") pod "2736624f-181c-4b3b-9088-3b08bde5c003" (UID: "2736624f-181c-4b3b-9088-3b08bde5c003"). InnerVolumeSpecName "kube-api-access-t9l5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:27:31.943376 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:31.943362 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2736624f-181c-4b3b-9088-3b08bde5c003-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2736624f-181c-4b3b-9088-3b08bde5c003" (UID: "2736624f-181c-4b3b-9088-3b08bde5c003"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 12:27:32.042733 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.042689 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2736624f-181c-4b3b-9088-3b08bde5c003-istio-podinfo\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:32.042733 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.042727 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-istio-token\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:32.042733 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.042737 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2736624f-181c-4b3b-9088-3b08bde5c003-istio-envoy\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:32.042733 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.042745 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9l5g\" (UniqueName: \"kubernetes.io/projected/2736624f-181c-4b3b-9088-3b08bde5c003-kube-api-access-t9l5g\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:32.591420 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.591388 2582 generic.go:356] "Generic (PLEG): container finished" podID="2736624f-181c-4b3b-9088-3b08bde5c003" containerID="14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd" exitCode=0 Sep 15 12:27:32.591648 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.591432 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" event={"ID":"2736624f-181c-4b3b-9088-3b08bde5c003","Type":"ContainerDied","Data":"14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd"} Sep 15 12:27:32.591648 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.591440 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" Sep 15 12:27:32.591648 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.591474 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6" event={"ID":"2736624f-181c-4b3b-9088-3b08bde5c003","Type":"ContainerDied","Data":"36e6c743fbd329fbc807b4f3461f29e03f51486ae389869b9de4eabedc3253d9"} Sep 15 12:27:32.591648 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.591510 2582 scope.go:117] "RemoveContainer" containerID="14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd" Sep 15 12:27:32.601791 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.601765 2582 scope.go:117] "RemoveContainer" containerID="8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1" Sep 15 12:27:32.610707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.610683 2582 scope.go:117] "RemoveContainer" containerID="14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd" Sep 15 12:27:32.611032 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:27:32.611011 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd\": container with ID starting with 14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd not found: ID does not exist" containerID="14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd" Sep 15 12:27:32.611087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.611043 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd"} err="failed to get container status \"14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd\": rpc error: code = NotFound desc = could not find container \"14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd\": container with ID starting with 14030088df333aec3dc1e05a224718a87e98d35c24cfc3a8603e156747ff51dd not found: ID does not exist" Sep 15 12:27:32.611087 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.611062 2582 scope.go:117] "RemoveContainer" containerID="8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1" Sep 15 12:27:32.611343 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:27:32.611323 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1\": container with ID starting with 8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1 not found: ID does not exist" containerID="8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1" Sep 15 12:27:32.611405 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.611353 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1"} err="failed to get container status \"8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1\": rpc error: code = NotFound desc = could not find container \"8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1\": container with ID starting with 8c9fb96b21230d325bb0486907d471a3163fe75dcf96f14dc1d4150fa7b5efc1 not found: ID does not exist" Sep 15 12:27:32.620501 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.620452 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6"] Sep 15 12:27:32.622661 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.622639 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-680229-openshift-default-54bf867d46-vkdp6"] Sep 15 12:27:32.712188 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:32.712151 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" path="/var/lib/kubelet/pods/2736624f-181c-4b3b-9088-3b08bde5c003/volumes" Sep 15 12:27:42.767360 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:42.767325 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct"] Sep 15 12:27:42.767893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:42.767570 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" podUID="ff42a837-75b8-457c-83d4-035aa491314e" containerName="payload-processing" containerID="cri-o://a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3" gracePeriod=30 Sep 15 12:27:42.775004 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:42.774973 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg"] Sep 15 12:27:42.775203 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:42.775183 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" podUID="59a90509-f186-4483-899b-c3812a2e2c42" containerName="payload-pre-processing" containerID="cri-o://c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a" gracePeriod=30 Sep 15 12:27:43.037779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.037752 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:27:43.068313 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.068279 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:27:43.157370 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.157335 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs5qg\" (UniqueName: \"kubernetes.io/projected/ff42a837-75b8-457c-83d4-035aa491314e-kube-api-access-gs5qg\") pod \"ff42a837-75b8-457c-83d4-035aa491314e\" (UID: \"ff42a837-75b8-457c-83d4-035aa491314e\") " Sep 15 12:27:43.157598 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.157389 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59a90509-f186-4483-899b-c3812a2e2c42-plugins-config-volume\") pod \"59a90509-f186-4483-899b-c3812a2e2c42\" (UID: \"59a90509-f186-4483-899b-c3812a2e2c42\") " Sep 15 12:27:43.157598 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.157416 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff42a837-75b8-457c-83d4-035aa491314e-plugins-config-volume\") pod \"ff42a837-75b8-457c-83d4-035aa491314e\" (UID: \"ff42a837-75b8-457c-83d4-035aa491314e\") " Sep 15 12:27:43.157598 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.157523 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km89l\" (UniqueName: \"kubernetes.io/projected/59a90509-f186-4483-899b-c3812a2e2c42-kube-api-access-km89l\") pod \"59a90509-f186-4483-899b-c3812a2e2c42\" (UID: \"59a90509-f186-4483-899b-c3812a2e2c42\") " Sep 15 12:27:43.157908 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.157857 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59a90509-f186-4483-899b-c3812a2e2c42-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "59a90509-f186-4483-899b-c3812a2e2c42" (UID: "59a90509-f186-4483-899b-c3812a2e2c42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:27:43.157908 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.157875 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff42a837-75b8-457c-83d4-035aa491314e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ff42a837-75b8-457c-83d4-035aa491314e" (UID: "ff42a837-75b8-457c-83d4-035aa491314e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:27:43.158324 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.158300 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59a90509-f186-4483-899b-c3812a2e2c42-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:43.158417 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.158331 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff42a837-75b8-457c-83d4-035aa491314e-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:43.160691 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.160661 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59a90509-f186-4483-899b-c3812a2e2c42-kube-api-access-km89l" (OuterVolumeSpecName: "kube-api-access-km89l") pod "59a90509-f186-4483-899b-c3812a2e2c42" (UID: "59a90509-f186-4483-899b-c3812a2e2c42"). InnerVolumeSpecName "kube-api-access-km89l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:27:43.160691 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.160674 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff42a837-75b8-457c-83d4-035aa491314e-kube-api-access-gs5qg" (OuterVolumeSpecName: "kube-api-access-gs5qg") pod "ff42a837-75b8-457c-83d4-035aa491314e" (UID: "ff42a837-75b8-457c-83d4-035aa491314e"). InnerVolumeSpecName "kube-api-access-gs5qg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:27:43.259363 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.259325 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km89l\" (UniqueName: \"kubernetes.io/projected/59a90509-f186-4483-899b-c3812a2e2c42-kube-api-access-km89l\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:43.259363 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.259355 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs5qg\" (UniqueName: \"kubernetes.io/projected/ff42a837-75b8-457c-83d4-035aa491314e-kube-api-access-gs5qg\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:27:43.656882 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.656844 2582 generic.go:356] "Generic (PLEG): container finished" podID="ff42a837-75b8-457c-83d4-035aa491314e" containerID="a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3" exitCode=0 Sep 15 12:27:43.657055 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.656886 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" Sep 15 12:27:43.657055 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.656923 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" event={"ID":"ff42a837-75b8-457c-83d4-035aa491314e","Type":"ContainerDied","Data":"a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3"} Sep 15 12:27:43.657055 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.657010 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct" event={"ID":"ff42a837-75b8-457c-83d4-035aa491314e","Type":"ContainerDied","Data":"8657624cd5e1e714c33f3b4c3aa704793ee05b67d681f12f10c32a982550de4f"} Sep 15 12:27:43.657055 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.657040 2582 scope.go:117] "RemoveContainer" containerID="a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3" Sep 15 12:27:43.658358 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.658337 2582 generic.go:356] "Generic (PLEG): container finished" podID="59a90509-f186-4483-899b-c3812a2e2c42" containerID="c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a" exitCode=0 Sep 15 12:27:43.658439 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.658395 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" Sep 15 12:27:43.658439 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.658425 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" event={"ID":"59a90509-f186-4483-899b-c3812a2e2c42","Type":"ContainerDied","Data":"c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a"} Sep 15 12:27:43.658557 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.658471 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg" event={"ID":"59a90509-f186-4483-899b-c3812a2e2c42","Type":"ContainerDied","Data":"1ada4ebff5b8a467e58de5daec01f583375a957423a5837e09830a5e100c3c2b"} Sep 15 12:27:43.666921 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.666900 2582 scope.go:117] "RemoveContainer" containerID="a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3" Sep 15 12:27:43.667218 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:27:43.667191 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3\": container with ID starting with a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3 not found: ID does not exist" containerID="a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3" Sep 15 12:27:43.667328 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.667220 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3"} err="failed to get container status \"a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3\": rpc error: code = NotFound desc = could not find container \"a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3\": container with ID starting with a51147cd0dda56588ce3d80abf26062afda3ef0bbc59a97065df2eba9e0dbcd3 not found: ID does not exist" Sep 15 12:27:43.667328 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.667242 2582 scope.go:117] "RemoveContainer" containerID="c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a" Sep 15 12:27:43.676198 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.676180 2582 scope.go:117] "RemoveContainer" containerID="c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a" Sep 15 12:27:43.676527 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:27:43.676501 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a\": container with ID starting with c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a not found: ID does not exist" containerID="c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a" Sep 15 12:27:43.676622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.676535 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a"} err="failed to get container status \"c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a\": rpc error: code = NotFound desc = could not find container \"c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a\": container with ID starting with c13718c1b66cd6c814b41c33a19601025a8af87f85d6cc30625a276a2304a14a not found: ID does not exist" Sep 15 12:27:43.682578 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.682552 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg"] Sep 15 12:27:43.687605 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.687583 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f1e118-95c46b4f8-cgndg"] Sep 15 12:27:43.706037 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.706003 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct"] Sep 15 12:27:43.709909 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:43.709886 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f1e118-6dc655d479-6n2ct"] Sep 15 12:27:44.711579 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:44.711544 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59a90509-f186-4483-899b-c3812a2e2c42" path="/var/lib/kubelet/pods/59a90509-f186-4483-899b-c3812a2e2c42/volumes" Sep 15 12:27:44.711941 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:27:44.711932 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff42a837-75b8-457c-83d4-035aa491314e" path="/var/lib/kubelet/pods/ff42a837-75b8-457c-83d4-035aa491314e/volumes" Sep 15 12:28:06.621835 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:06.621802 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr"] Sep 15 12:28:06.622472 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:06.622023 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" podUID="6aecbcd6-b459-419b-8a7e-ccc9ccb7b931" containerName="payload-processing" containerID="cri-o://06a2483c05f7f577fcebab08f53099ef507ca144ed42b57e3fcce5f65117f5ee" gracePeriod=30 Sep 15 12:28:06.799753 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:06.799722 2582 generic.go:356] "Generic (PLEG): container finished" podID="6aecbcd6-b459-419b-8a7e-ccc9ccb7b931" containerID="06a2483c05f7f577fcebab08f53099ef507ca144ed42b57e3fcce5f65117f5ee" exitCode=0 Sep 15 12:28:06.799905 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:06.799781 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" event={"ID":"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931","Type":"ContainerDied","Data":"06a2483c05f7f577fcebab08f53099ef507ca144ed42b57e3fcce5f65117f5ee"} Sep 15 12:28:06.892958 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:06.892929 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:28:07.012742 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.012707 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ql28\" (UniqueName: \"kubernetes.io/projected/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-kube-api-access-8ql28\") pod \"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931\" (UID: \"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931\") " Sep 15 12:28:07.012947 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.012851 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-plugins-config-volume\") pod \"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931\" (UID: \"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931\") " Sep 15 12:28:07.013379 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.013345 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6aecbcd6-b459-419b-8a7e-ccc9ccb7b931" (UID: "6aecbcd6-b459-419b-8a7e-ccc9ccb7b931"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:28:07.015598 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.015568 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-kube-api-access-8ql28" (OuterVolumeSpecName: "kube-api-access-8ql28") pod "6aecbcd6-b459-419b-8a7e-ccc9ccb7b931" (UID: "6aecbcd6-b459-419b-8a7e-ccc9ccb7b931"). InnerVolumeSpecName "kube-api-access-8ql28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:28:07.114469 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.114432 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ql28\" (UniqueName: \"kubernetes.io/projected/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-kube-api-access-8ql28\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:07.114469 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.114464 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:07.806972 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.806942 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" Sep 15 12:28:07.807386 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.806982 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr" event={"ID":"6aecbcd6-b459-419b-8a7e-ccc9ccb7b931","Type":"ContainerDied","Data":"26a514db3ee08362bd3ca1d110611899545b70287643b1b0bb383dc2d4f3edb5"} Sep 15 12:28:07.807386 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.807055 2582 scope.go:117] "RemoveContainer" containerID="06a2483c05f7f577fcebab08f53099ef507ca144ed42b57e3fcce5f65117f5ee" Sep 15 12:28:07.835921 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.835889 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr"] Sep 15 12:28:07.846541 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:07.846513 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-eb9bbe-5689b9f55f-2whzr"] Sep 15 12:28:08.711696 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:08.711661 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6aecbcd6-b459-419b-8a7e-ccc9ccb7b931" path="/var/lib/kubelet/pods/6aecbcd6-b459-419b-8a7e-ccc9ccb7b931/volumes" Sep 15 12:28:25.265417 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.265378 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n"] Sep 15 12:28:25.265941 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.265663 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" podUID="82b92dfd-ad27-4eb7-9cc3-2166dd4cb306" containerName="payload-processing" containerID="cri-o://bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629" gracePeriod=30 Sep 15 12:28:25.278975 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.278943 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv"] Sep 15 12:28:25.279241 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.279195 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" podUID="2230fc2b-d18c-4acc-8f0a-60a5138c125b" containerName="payload-pre-processing" containerID="cri-o://30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039" gracePeriod=30 Sep 15 12:28:25.554049 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.554020 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:28:25.586263 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.586234 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:28:25.704721 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.704673 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7ddp\" (UniqueName: \"kubernetes.io/projected/2230fc2b-d18c-4acc-8f0a-60a5138c125b-kube-api-access-s7ddp\") pod \"2230fc2b-d18c-4acc-8f0a-60a5138c125b\" (UID: \"2230fc2b-d18c-4acc-8f0a-60a5138c125b\") " Sep 15 12:28:25.704900 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.704729 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2kfd\" (UniqueName: \"kubernetes.io/projected/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-kube-api-access-b2kfd\") pod \"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306\" (UID: \"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306\") " Sep 15 12:28:25.704900 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.704755 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-plugins-config-volume\") pod \"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306\" (UID: \"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306\") " Sep 15 12:28:25.704985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.704916 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2230fc2b-d18c-4acc-8f0a-60a5138c125b-plugins-config-volume\") pod \"2230fc2b-d18c-4acc-8f0a-60a5138c125b\" (UID: \"2230fc2b-d18c-4acc-8f0a-60a5138c125b\") " Sep 15 12:28:25.705258 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.705226 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82b92dfd-ad27-4eb7-9cc3-2166dd4cb306" (UID: "82b92dfd-ad27-4eb7-9cc3-2166dd4cb306"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:28:25.705407 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.705258 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2230fc2b-d18c-4acc-8f0a-60a5138c125b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2230fc2b-d18c-4acc-8f0a-60a5138c125b" (UID: "2230fc2b-d18c-4acc-8f0a-60a5138c125b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:28:25.707057 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.707035 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2230fc2b-d18c-4acc-8f0a-60a5138c125b-kube-api-access-s7ddp" (OuterVolumeSpecName: "kube-api-access-s7ddp") pod "2230fc2b-d18c-4acc-8f0a-60a5138c125b" (UID: "2230fc2b-d18c-4acc-8f0a-60a5138c125b"). InnerVolumeSpecName "kube-api-access-s7ddp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:28:25.707160 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.707138 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-kube-api-access-b2kfd" (OuterVolumeSpecName: "kube-api-access-b2kfd") pod "82b92dfd-ad27-4eb7-9cc3-2166dd4cb306" (UID: "82b92dfd-ad27-4eb7-9cc3-2166dd4cb306"). InnerVolumeSpecName "kube-api-access-b2kfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:28:25.806885 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.806760 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2230fc2b-d18c-4acc-8f0a-60a5138c125b-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:25.807064 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.806956 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7ddp\" (UniqueName: \"kubernetes.io/projected/2230fc2b-d18c-4acc-8f0a-60a5138c125b-kube-api-access-s7ddp\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:25.807064 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.806992 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2kfd\" (UniqueName: \"kubernetes.io/projected/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-kube-api-access-b2kfd\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:25.807064 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.807012 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:25.907245 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.907210 2582 generic.go:356] "Generic (PLEG): container finished" podID="2230fc2b-d18c-4acc-8f0a-60a5138c125b" containerID="30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039" exitCode=0 Sep 15 12:28:25.907458 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.907263 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" Sep 15 12:28:25.907458 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.907298 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" event={"ID":"2230fc2b-d18c-4acc-8f0a-60a5138c125b","Type":"ContainerDied","Data":"30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039"} Sep 15 12:28:25.907458 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.907358 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv" event={"ID":"2230fc2b-d18c-4acc-8f0a-60a5138c125b","Type":"ContainerDied","Data":"ebc1f779a5d87abf3704f80c38f09ce81f4654c31198ec13ab806124783843ee"} Sep 15 12:28:25.907458 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.907392 2582 scope.go:117] "RemoveContainer" containerID="30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039" Sep 15 12:28:25.908751 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.908725 2582 generic.go:356] "Generic (PLEG): container finished" podID="82b92dfd-ad27-4eb7-9cc3-2166dd4cb306" containerID="bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629" exitCode=0 Sep 15 12:28:25.908850 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.908760 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" Sep 15 12:28:25.908850 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.908752 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" event={"ID":"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306","Type":"ContainerDied","Data":"bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629"} Sep 15 12:28:25.908850 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.908802 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n" event={"ID":"82b92dfd-ad27-4eb7-9cc3-2166dd4cb306","Type":"ContainerDied","Data":"d4a90da591105cec316671a24c23a170eadcb9471b509602980368b767f5a1ee"} Sep 15 12:28:25.918284 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.918256 2582 scope.go:117] "RemoveContainer" containerID="30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039" Sep 15 12:28:25.918691 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:28:25.918663 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039\": container with ID starting with 30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039 not found: ID does not exist" containerID="30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039" Sep 15 12:28:25.918781 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.918704 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039"} err="failed to get container status \"30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039\": rpc error: code = NotFound desc = could not find container \"30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039\": container with ID starting with 30b2477a2d9412972b080c8f60b636d475a41d2f086f0d90e72175a4970a3039 not found: ID does not exist" Sep 15 12:28:25.918781 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.918722 2582 scope.go:117] "RemoveContainer" containerID="bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629" Sep 15 12:28:25.927348 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.927330 2582 scope.go:117] "RemoveContainer" containerID="bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629" Sep 15 12:28:25.927671 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:28:25.927654 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629\": container with ID starting with bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629 not found: ID does not exist" containerID="bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629" Sep 15 12:28:25.927736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.927677 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629"} err="failed to get container status \"bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629\": rpc error: code = NotFound desc = could not find container \"bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629\": container with ID starting with bbee8bbfe41bc806f66584332a8a4f610e90b1c7590c9f3662de5e8657340629 not found: ID does not exist" Sep 15 12:28:25.940049 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.940016 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n"] Sep 15 12:28:25.942270 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.942251 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-58b0fd-7977866c47-9fd8n"] Sep 15 12:28:25.962510 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.962456 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv"] Sep 15 12:28:25.970688 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:25.970663 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-58b0fd-6cc665674-hf8tv"] Sep 15 12:28:26.712679 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:26.712653 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2230fc2b-d18c-4acc-8f0a-60a5138c125b" path="/var/lib/kubelet/pods/2230fc2b-d18c-4acc-8f0a-60a5138c125b/volumes" Sep 15 12:28:26.713056 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:26.713042 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82b92dfd-ad27-4eb7-9cc3-2166dd4cb306" path="/var/lib/kubelet/pods/82b92dfd-ad27-4eb7-9cc3-2166dd4cb306/volumes" Sep 15 12:28:28.925419 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:28.925384 2582 scope.go:117] "RemoveContainer" containerID="0fbc1734a84712a9a9fc7c46501d4ea94f40c7b8588619c741d687fa7cce84ca" Sep 15 12:28:31.692323 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.692287 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8"] Sep 15 12:28:31.692931 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.692910 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3444551-04fb-482e-9f2f-5a74395afc19" containerName="payload-processing" Sep 15 12:28:31.693020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.692934 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3444551-04fb-482e-9f2f-5a74395afc19" containerName="payload-processing" Sep 15 12:28:31.693020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.692951 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8ad94c8-d45e-435b-aa5b-ead9d00f802e" containerName="payload-pre-processing" Sep 15 12:28:31.693020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.692961 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8ad94c8-d45e-435b-aa5b-ead9d00f802e" containerName="payload-pre-processing" Sep 15 12:28:31.693020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.692979 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aecbcd6-b459-419b-8a7e-ccc9ccb7b931" containerName="payload-processing" Sep 15 12:28:31.693020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.692988 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aecbcd6-b459-419b-8a7e-ccc9ccb7b931" containerName="payload-processing" Sep 15 12:28:31.693020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693004 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59a90509-f186-4483-899b-c3812a2e2c42" containerName="payload-pre-processing" Sep 15 12:28:31.693020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693012 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="59a90509-f186-4483-899b-c3812a2e2c42" containerName="payload-pre-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693025 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b92dfd-ad27-4eb7-9cc3-2166dd4cb306" containerName="payload-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693034 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b92dfd-ad27-4eb7-9cc3-2166dd4cb306" containerName="payload-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693049 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693058 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693072 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2230fc2b-d18c-4acc-8f0a-60a5138c125b" containerName="payload-pre-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693081 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2230fc2b-d18c-4acc-8f0a-60a5138c125b" containerName="payload-pre-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693091 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff42a837-75b8-457c-83d4-035aa491314e" containerName="payload-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693100 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff42a837-75b8-457c-83d4-035aa491314e" containerName="payload-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693207 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693218 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff42a837-75b8-457c-83d4-035aa491314e" containerName="payload-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693228 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="59a90509-f186-4483-899b-c3812a2e2c42" containerName="payload-pre-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693239 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3444551-04fb-482e-9f2f-5a74395afc19" containerName="payload-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693247 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="82b92dfd-ad27-4eb7-9cc3-2166dd4cb306" containerName="payload-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693256 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2230fc2b-d18c-4acc-8f0a-60a5138c125b" containerName="payload-pre-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693265 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aecbcd6-b459-419b-8a7e-ccc9ccb7b931" containerName="payload-processing" Sep 15 12:28:31.693364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.693274 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8ad94c8-d45e-435b-aa5b-ead9d00f802e" containerName="payload-pre-processing" Sep 15 12:28:31.698403 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.698380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" Sep 15 12:28:31.701516 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.701479 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:28:31.701643 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.701602 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:28:31.740439 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.740400 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8"] Sep 15 12:28:31.869059 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.869016 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/500781b9-a0e2-4a66-8b4c-8e12611e920d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8\" (UID: \"500781b9-a0e2-4a66-8b4c-8e12611e920d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" Sep 15 12:28:31.970076 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.970040 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/500781b9-a0e2-4a66-8b4c-8e12611e920d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8\" (UID: \"500781b9-a0e2-4a66-8b4c-8e12611e920d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" Sep 15 12:28:31.970727 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:31.970709 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/500781b9-a0e2-4a66-8b4c-8e12611e920d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8\" (UID: \"500781b9-a0e2-4a66-8b4c-8e12611e920d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" Sep 15 12:28:32.008611 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:32.008264 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" Sep 15 12:28:32.288476 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:32.288446 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8"] Sep 15 12:28:32.289431 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:28:32.289408 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod500781b9_a0e2_4a66_8b4c_8e12611e920d.slice/crio-89d1cc645477d4b00eb73f3e7babbd34921e55b76d99e4605b8b98d606b000d4 WatchSource:0}: Error finding container 89d1cc645477d4b00eb73f3e7babbd34921e55b76d99e4605b8b98d606b000d4: Status 404 returned error can't find the container with id 89d1cc645477d4b00eb73f3e7babbd34921e55b76d99e4605b8b98d606b000d4 Sep 15 12:28:32.948537 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:32.948416 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" event={"ID":"500781b9-a0e2-4a66-8b4c-8e12611e920d","Type":"ContainerStarted","Data":"9a2ee27f02eb28ea7196cb43509d7b488987cff40cd3b58e9b80f0ed08c62aa3"} Sep 15 12:28:32.948537 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:32.948497 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" event={"ID":"500781b9-a0e2-4a66-8b4c-8e12611e920d","Type":"ContainerStarted","Data":"89d1cc645477d4b00eb73f3e7babbd34921e55b76d99e4605b8b98d606b000d4"} Sep 15 12:28:32.967817 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:32.967764 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" podStartSLOduration=1.967748777 podStartE2EDuration="1.967748777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:28:32.9665065 +0000 UTC m=+1564.723611564" watchObservedRunningTime="2026-09-15 12:28:32.967748777 +0000 UTC m=+1564.724853847" Sep 15 12:28:33.956117 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:33.956083 2582 generic.go:356] "Generic (PLEG): container finished" podID="500781b9-a0e2-4a66-8b4c-8e12611e920d" containerID="9a2ee27f02eb28ea7196cb43509d7b488987cff40cd3b58e9b80f0ed08c62aa3" exitCode=0 Sep 15 12:28:33.956555 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:33.956179 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" event={"ID":"500781b9-a0e2-4a66-8b4c-8e12611e920d","Type":"ContainerDied","Data":"9a2ee27f02eb28ea7196cb43509d7b488987cff40cd3b58e9b80f0ed08c62aa3"} Sep 15 12:28:35.104385 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:35.104362 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" Sep 15 12:28:35.196148 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:35.196112 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/500781b9-a0e2-4a66-8b4c-8e12611e920d-maas-api-service-ca\") pod \"500781b9-a0e2-4a66-8b4c-8e12611e920d\" (UID: \"500781b9-a0e2-4a66-8b4c-8e12611e920d\") " Sep 15 12:28:35.196518 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:35.196459 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/500781b9-a0e2-4a66-8b4c-8e12611e920d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "500781b9-a0e2-4a66-8b4c-8e12611e920d" (UID: "500781b9-a0e2-4a66-8b4c-8e12611e920d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:28:35.297660 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:35.297572 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/500781b9-a0e2-4a66-8b4c-8e12611e920d-maas-api-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:35.968723 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:35.968683 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" Sep 15 12:28:35.968904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:35.968688 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8" event={"ID":"500781b9-a0e2-4a66-8b4c-8e12611e920d","Type":"ContainerDied","Data":"89d1cc645477d4b00eb73f3e7babbd34921e55b76d99e4605b8b98d606b000d4"} Sep 15 12:28:35.968904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:35.968823 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89d1cc645477d4b00eb73f3e7babbd34921e55b76d99e4605b8b98d606b000d4" Sep 15 12:28:44.623217 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:44.623182 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq"] Sep 15 12:28:44.623743 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:44.623407 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" podUID="2d0c377e-10d0-491c-9825-93311a12a414" containerName="payload-processing" containerID="cri-o://6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c" gracePeriod=30 Sep 15 12:28:44.886418 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:44.886338 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:28:44.990186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:44.990149 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d0c377e-10d0-491c-9825-93311a12a414-plugins-config-volume\") pod \"2d0c377e-10d0-491c-9825-93311a12a414\" (UID: \"2d0c377e-10d0-491c-9825-93311a12a414\") " Sep 15 12:28:44.990186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:44.990189 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbwp5\" (UniqueName: \"kubernetes.io/projected/2d0c377e-10d0-491c-9825-93311a12a414-kube-api-access-tbwp5\") pod \"2d0c377e-10d0-491c-9825-93311a12a414\" (UID: \"2d0c377e-10d0-491c-9825-93311a12a414\") " Sep 15 12:28:44.990657 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:44.990627 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d0c377e-10d0-491c-9825-93311a12a414-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2d0c377e-10d0-491c-9825-93311a12a414" (UID: "2d0c377e-10d0-491c-9825-93311a12a414"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:28:44.992860 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:44.992823 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d0c377e-10d0-491c-9825-93311a12a414-kube-api-access-tbwp5" (OuterVolumeSpecName: "kube-api-access-tbwp5") pod "2d0c377e-10d0-491c-9825-93311a12a414" (UID: "2d0c377e-10d0-491c-9825-93311a12a414"). InnerVolumeSpecName "kube-api-access-tbwp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:28:45.016992 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.016957 2582 generic.go:356] "Generic (PLEG): container finished" podID="2d0c377e-10d0-491c-9825-93311a12a414" containerID="6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c" exitCode=0 Sep 15 12:28:45.017156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.017009 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" event={"ID":"2d0c377e-10d0-491c-9825-93311a12a414","Type":"ContainerDied","Data":"6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c"} Sep 15 12:28:45.017156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.017033 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" Sep 15 12:28:45.017156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.017102 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq" event={"ID":"2d0c377e-10d0-491c-9825-93311a12a414","Type":"ContainerDied","Data":"6e5f9c2e260b5512acbdd3729c4c4a5bc281fe68bb9e58ebe34986e97254b808"} Sep 15 12:28:45.017156 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.017127 2582 scope.go:117] "RemoveContainer" containerID="6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c" Sep 15 12:28:45.028917 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.028817 2582 scope.go:117] "RemoveContainer" containerID="6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c" Sep 15 12:28:45.029632 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:28:45.029593 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c\": container with ID starting with 6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c not found: ID does not exist" containerID="6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c" Sep 15 12:28:45.029746 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.029635 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c"} err="failed to get container status \"6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c\": rpc error: code = NotFound desc = could not find container \"6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c\": container with ID starting with 6915a9236690a5b8483ea9a82ef95527d161acb91724f0e54ea24f7fdefc311c not found: ID does not exist" Sep 15 12:28:45.057207 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.057171 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq"] Sep 15 12:28:45.061596 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.061570 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ba366c-79c79dd677-8xlqq"] Sep 15 12:28:45.091852 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.091822 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d0c377e-10d0-491c-9825-93311a12a414-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:45.091852 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.091849 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbwp5\" (UniqueName: \"kubernetes.io/projected/2d0c377e-10d0-491c-9825-93311a12a414-kube-api-access-tbwp5\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:45.199046 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.198963 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz"] Sep 15 12:28:45.199210 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.199184 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" podUID="4eec96e7-203a-47e1-9338-b7163ab056d8" containerName="payload-processing" containerID="cri-o://17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8" gracePeriod=30 Sep 15 12:28:45.206950 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.206920 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j"] Sep 15 12:28:45.207218 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.207171 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" podUID="c59d4179-f5d4-4175-83e4-ea3d4212a5ef" containerName="payload-pre-processing" containerID="cri-o://39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b" gracePeriod=30 Sep 15 12:28:45.474052 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.474023 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:28:45.504974 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.504935 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:28:45.598556 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.598516 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-plugins-config-volume\") pod \"c59d4179-f5d4-4175-83e4-ea3d4212a5ef\" (UID: \"c59d4179-f5d4-4175-83e4-ea3d4212a5ef\") " Sep 15 12:28:45.598718 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.598601 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48qpq\" (UniqueName: \"kubernetes.io/projected/4eec96e7-203a-47e1-9338-b7163ab056d8-kube-api-access-48qpq\") pod \"4eec96e7-203a-47e1-9338-b7163ab056d8\" (UID: \"4eec96e7-203a-47e1-9338-b7163ab056d8\") " Sep 15 12:28:45.598718 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.598652 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eec96e7-203a-47e1-9338-b7163ab056d8-plugins-config-volume\") pod \"4eec96e7-203a-47e1-9338-b7163ab056d8\" (UID: \"4eec96e7-203a-47e1-9338-b7163ab056d8\") " Sep 15 12:28:45.598718 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.598687 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmwrk\" (UniqueName: \"kubernetes.io/projected/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-kube-api-access-jmwrk\") pod \"c59d4179-f5d4-4175-83e4-ea3d4212a5ef\" (UID: \"c59d4179-f5d4-4175-83e4-ea3d4212a5ef\") " Sep 15 12:28:45.598901 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.598872 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c59d4179-f5d4-4175-83e4-ea3d4212a5ef" (UID: "c59d4179-f5d4-4175-83e4-ea3d4212a5ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:28:45.599027 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.599003 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eec96e7-203a-47e1-9338-b7163ab056d8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4eec96e7-203a-47e1-9338-b7163ab056d8" (UID: "4eec96e7-203a-47e1-9338-b7163ab056d8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:28:45.599098 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.599034 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:45.600914 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.600893 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-kube-api-access-jmwrk" (OuterVolumeSpecName: "kube-api-access-jmwrk") pod "c59d4179-f5d4-4175-83e4-ea3d4212a5ef" (UID: "c59d4179-f5d4-4175-83e4-ea3d4212a5ef"). InnerVolumeSpecName "kube-api-access-jmwrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:28:45.601006 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.600933 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eec96e7-203a-47e1-9338-b7163ab056d8-kube-api-access-48qpq" (OuterVolumeSpecName: "kube-api-access-48qpq") pod "4eec96e7-203a-47e1-9338-b7163ab056d8" (UID: "4eec96e7-203a-47e1-9338-b7163ab056d8"). InnerVolumeSpecName "kube-api-access-48qpq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:28:45.605740 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.605710 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8"] Sep 15 12:28:45.613389 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.613365 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9ba48f-nljr8"] Sep 15 12:28:45.699692 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.699662 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmwrk\" (UniqueName: \"kubernetes.io/projected/c59d4179-f5d4-4175-83e4-ea3d4212a5ef-kube-api-access-jmwrk\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:45.699692 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.699690 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48qpq\" (UniqueName: \"kubernetes.io/projected/4eec96e7-203a-47e1-9338-b7163ab056d8-kube-api-access-48qpq\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:45.699692 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:45.699701 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eec96e7-203a-47e1-9338-b7163ab056d8-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:46.023740 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.023710 2582 generic.go:356] "Generic (PLEG): container finished" podID="c59d4179-f5d4-4175-83e4-ea3d4212a5ef" containerID="39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b" exitCode=0 Sep 15 12:28:46.023934 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.023764 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" Sep 15 12:28:46.023934 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.023812 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" event={"ID":"c59d4179-f5d4-4175-83e4-ea3d4212a5ef","Type":"ContainerDied","Data":"39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b"} Sep 15 12:28:46.023934 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.023864 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j" event={"ID":"c59d4179-f5d4-4175-83e4-ea3d4212a5ef","Type":"ContainerDied","Data":"9478d6e8b6a962575cbee026d28310578b6dff12f5de85704d582b44240d03e0"} Sep 15 12:28:46.023934 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.023895 2582 scope.go:117] "RemoveContainer" containerID="39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b" Sep 15 12:28:46.025423 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.025393 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eec96e7-203a-47e1-9338-b7163ab056d8" containerID="17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8" exitCode=0 Sep 15 12:28:46.025581 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.025435 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" Sep 15 12:28:46.025581 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.025502 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" event={"ID":"4eec96e7-203a-47e1-9338-b7163ab056d8","Type":"ContainerDied","Data":"17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8"} Sep 15 12:28:46.025581 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.025560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz" event={"ID":"4eec96e7-203a-47e1-9338-b7163ab056d8","Type":"ContainerDied","Data":"e28f040b848a75a701419ba6d9dce25b30fed8998977d536cf6a04198aeb2f0f"} Sep 15 12:28:46.033604 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.033578 2582 scope.go:117] "RemoveContainer" containerID="39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b" Sep 15 12:28:46.033999 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:28:46.033979 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b\": container with ID starting with 39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b not found: ID does not exist" containerID="39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b" Sep 15 12:28:46.034097 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.034006 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b"} err="failed to get container status \"39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b\": rpc error: code = NotFound desc = could not find container \"39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b\": container with ID starting with 39be3fa3a7d933a023954a1a284e467b520a1e2b123c42078b013e4d17da6b4b not found: ID does not exist" Sep 15 12:28:46.034097 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.034024 2582 scope.go:117] "RemoveContainer" containerID="17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8" Sep 15 12:28:46.042633 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.042615 2582 scope.go:117] "RemoveContainer" containerID="17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8" Sep 15 12:28:46.042914 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:28:46.042892 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8\": container with ID starting with 17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8 not found: ID does not exist" containerID="17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8" Sep 15 12:28:46.042985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.042922 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8"} err="failed to get container status \"17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8\": rpc error: code = NotFound desc = could not find container \"17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8\": container with ID starting with 17c52192e87044e2074e4430ef9f8ac67a8757f7c79b59501366cf5b94aebfb8 not found: ID does not exist" Sep 15 12:28:46.054562 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.054528 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz"] Sep 15 12:28:46.059978 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.059952 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-9ba48f-6d989967f5-486rz"] Sep 15 12:28:46.076062 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.076037 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j"] Sep 15 12:28:46.077800 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.077780 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9ba48f-8447c676f4-gmk2j"] Sep 15 12:28:46.712331 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.712302 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d0c377e-10d0-491c-9825-93311a12a414" path="/var/lib/kubelet/pods/2d0c377e-10d0-491c-9825-93311a12a414/volumes" Sep 15 12:28:46.712728 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.712682 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eec96e7-203a-47e1-9338-b7163ab056d8" path="/var/lib/kubelet/pods/4eec96e7-203a-47e1-9338-b7163ab056d8/volumes" Sep 15 12:28:46.712987 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.712972 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="500781b9-a0e2-4a66-8b4c-8e12611e920d" path="/var/lib/kubelet/pods/500781b9-a0e2-4a66-8b4c-8e12611e920d/volumes" Sep 15 12:28:46.713290 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:46.713279 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c59d4179-f5d4-4175-83e4-ea3d4212a5ef" path="/var/lib/kubelet/pods/c59d4179-f5d4-4175-83e4-ea3d4212a5ef/volumes" Sep 15 12:28:53.578324 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:53.578289 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh"] Sep 15 12:28:53.578888 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:53.578504 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" containerID="cri-o://b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d" gracePeriod=30 Sep 15 12:28:58.839877 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.839847 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:28:58.929273 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929185 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-envoy\") pod \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " Sep 15 12:28:58.929273 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929228 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-credential-socket\") pod \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " Sep 15 12:28:58.929544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929280 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-data\") pod \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " Sep 15 12:28:58.929544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929314 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istiod-ca-cert\") pod \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " Sep 15 12:28:58.929544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929347 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-socket\") pod \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " Sep 15 12:28:58.929544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929375 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-podinfo\") pod \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " Sep 15 12:28:58.929544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929404 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-token\") pod \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " Sep 15 12:28:58.929544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929459 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-certs\") pod \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " Sep 15 12:28:58.929862 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929554 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgqhv\" (UniqueName: \"kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-kube-api-access-hgqhv\") pod \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\" (UID: \"0c5aff6b-cd5e-4092-b33f-9574119b8d82\") " Sep 15 12:28:58.929862 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929654 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0c5aff6b-cd5e-4092-b33f-9574119b8d82" (UID: "0c5aff6b-cd5e-4092-b33f-9574119b8d82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:28:58.929862 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929740 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-data" (OuterVolumeSpecName: "istio-data") pod "0c5aff6b-cd5e-4092-b33f-9574119b8d82" (UID: "0c5aff6b-cd5e-4092-b33f-9574119b8d82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:28:58.929999 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929854 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0c5aff6b-cd5e-4092-b33f-9574119b8d82" (UID: "0c5aff6b-cd5e-4092-b33f-9574119b8d82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:28:58.929999 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929871 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0c5aff6b-cd5e-4092-b33f-9574119b8d82" (UID: "0c5aff6b-cd5e-4092-b33f-9574119b8d82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:28:58.930104 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.929994 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0c5aff6b-cd5e-4092-b33f-9574119b8d82" (UID: "0c5aff6b-cd5e-4092-b33f-9574119b8d82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:28:58.930104 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.930055 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-certs\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:58.930104 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.930074 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-data\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:58.930104 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.930088 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istiod-ca-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:58.930104 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.930100 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-workload-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:58.931740 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.931715 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0c5aff6b-cd5e-4092-b33f-9574119b8d82" (UID: "0c5aff6b-cd5e-4092-b33f-9574119b8d82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:28:58.931869 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.931836 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-token" (OuterVolumeSpecName: "istio-token") pod "0c5aff6b-cd5e-4092-b33f-9574119b8d82" (UID: "0c5aff6b-cd5e-4092-b33f-9574119b8d82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:28:58.931938 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.931914 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-kube-api-access-hgqhv" (OuterVolumeSpecName: "kube-api-access-hgqhv") pod "0c5aff6b-cd5e-4092-b33f-9574119b8d82" (UID: "0c5aff6b-cd5e-4092-b33f-9574119b8d82"). InnerVolumeSpecName "kube-api-access-hgqhv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:28:58.932025 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:58.932007 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0c5aff6b-cd5e-4092-b33f-9574119b8d82" (UID: "0c5aff6b-cd5e-4092-b33f-9574119b8d82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 12:28:59.031502 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.031453 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgqhv\" (UniqueName: \"kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-kube-api-access-hgqhv\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:59.031680 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.031513 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-envoy\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:59.031680 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.031524 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5aff6b-cd5e-4092-b33f-9574119b8d82-credential-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:59.031680 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.031533 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-podinfo\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:59.031680 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.031542 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c5aff6b-cd5e-4092-b33f-9574119b8d82-istio-token\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:28:59.100365 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.100334 2582 generic.go:356] "Generic (PLEG): container finished" podID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerID="b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d" exitCode=0 Sep 15 12:28:59.100563 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.100383 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" Sep 15 12:28:59.100563 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.100419 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" event={"ID":"0c5aff6b-cd5e-4092-b33f-9574119b8d82","Type":"ContainerDied","Data":"b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d"} Sep 15 12:28:59.100563 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.100475 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh" event={"ID":"0c5aff6b-cd5e-4092-b33f-9574119b8d82","Type":"ContainerDied","Data":"b49ce3455d7f66d9f9d47a69c46fab0a7b0e5e38f365ba1e39f27a3ab568674f"} Sep 15 12:28:59.100563 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.100512 2582 scope.go:117] "RemoveContainer" containerID="b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d" Sep 15 12:28:59.110519 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.110500 2582 scope.go:117] "RemoveContainer" containerID="7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a" Sep 15 12:28:59.118471 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.118455 2582 scope.go:117] "RemoveContainer" containerID="b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d" Sep 15 12:28:59.118817 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:28:59.118797 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d\": container with ID starting with b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d not found: ID does not exist" containerID="b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d" Sep 15 12:28:59.118899 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.118824 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d"} err="failed to get container status \"b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d\": rpc error: code = NotFound desc = could not find container \"b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d\": container with ID starting with b1b44515e643eca1d2d311830b92f940f724c1f45ed0d5d4495394c10230de4d not found: ID does not exist" Sep 15 12:28:59.118899 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.118841 2582 scope.go:117] "RemoveContainer" containerID="7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a" Sep 15 12:28:59.119088 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:28:59.119070 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a\": container with ID starting with 7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a not found: ID does not exist" containerID="7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a" Sep 15 12:28:59.119141 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.119097 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a"} err="failed to get container status \"7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a\": rpc error: code = NotFound desc = could not find container \"7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a\": container with ID starting with 7a92691f0c50dfcdaa3808f8c3dbc8a40f841423813a0b3f6785e96e91e93b2a not found: ID does not exist" Sep 15 12:28:59.124702 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.124677 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh"] Sep 15 12:28:59.129887 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:28:59.129865 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba366c-openshift-default-6c6d4b5b7d-dp8vh"] Sep 15 12:29:00.711683 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:29:00.711645 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" path="/var/lib/kubelet/pods/0c5aff6b-cd5e-4092-b33f-9574119b8d82/volumes" Sep 15 12:30:00.139366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139332 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm"] Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139768 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eec96e7-203a-47e1-9338-b7163ab056d8" containerName="payload-processing" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139781 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eec96e7-203a-47e1-9338-b7163ab056d8" containerName="payload-processing" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139792 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d0c377e-10d0-491c-9825-93311a12a414" containerName="payload-processing" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139798 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d0c377e-10d0-491c-9825-93311a12a414" containerName="payload-processing" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139807 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139813 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139820 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c59d4179-f5d4-4175-83e4-ea3d4212a5ef" containerName="payload-pre-processing" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139825 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c59d4179-f5d4-4175-83e4-ea3d4212a5ef" containerName="payload-pre-processing" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139837 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139843 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139848 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500781b9-a0e2-4a66-8b4c-8e12611e920d" containerName="revoke-keys" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139853 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="500781b9-a0e2-4a66-8b4c-8e12611e920d" containerName="revoke-keys" Sep 15 12:30:00.139903 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139910 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d0c377e-10d0-491c-9825-93311a12a414" containerName="payload-processing" Sep 15 12:30:00.140357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139918 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2736624f-181c-4b3b-9088-3b08bde5c003" containerName="istio-proxy" Sep 15 12:30:00.140357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139925 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eec96e7-203a-47e1-9338-b7163ab056d8" containerName="payload-processing" Sep 15 12:30:00.140357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139932 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" Sep 15 12:30:00.140357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139939 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="500781b9-a0e2-4a66-8b4c-8e12611e920d" containerName="revoke-keys" Sep 15 12:30:00.140357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139944 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" Sep 15 12:30:00.140357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.139950 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c59d4179-f5d4-4175-83e4-ea3d4212a5ef" containerName="payload-pre-processing" Sep 15 12:30:00.142873 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.142856 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq"] Sep 15 12:30:00.142992 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.142971 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm" Sep 15 12:30:00.143227 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.143216 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" Sep 15 12:30:00.143266 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.143230 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c5aff6b-cd5e-4092-b33f-9574119b8d82" containerName="istio-proxy" Sep 15 12:30:00.145440 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.145416 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:30:00.146232 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.146210 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq" Sep 15 12:30:00.153143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.152003 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq"] Sep 15 12:30:00.154616 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.154591 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm"] Sep 15 12:30:00.156933 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.156914 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm" Sep 15 12:30:00.166375 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.166355 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq" Sep 15 12:30:00.436893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.436864 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm"] Sep 15 12:30:00.439469 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:30:00.439436 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62512f37_af98_4011_9d99_fbe7306ec1ce.slice/crio-68551121ce6aaabcfad2c301948e4d328656d637c01e8289d721711089c463e8 WatchSource:0}: Error finding container 68551121ce6aaabcfad2c301948e4d328656d637c01e8289d721711089c463e8: Status 404 returned error can't find the container with id 68551121ce6aaabcfad2c301948e4d328656d637c01e8289d721711089c463e8 Sep 15 12:30:00.446743 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:00.446659 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq"] Sep 15 12:30:00.449078 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:30:00.449054 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a7581f_3035_4803_881d_a884a9a1111d.slice/crio-6c6093440bcc8d3baedc423cb9f19a844fdfc254629859e21702f285400e39c2 WatchSource:0}: Error finding container 6c6093440bcc8d3baedc423cb9f19a844fdfc254629859e21702f285400e39c2: Status 404 returned error can't find the container with id 6c6093440bcc8d3baedc423cb9f19a844fdfc254629859e21702f285400e39c2 Sep 15 12:30:01.412968 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:01.412931 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm" event={"ID":"62512f37-af98-4011-9d99-fbe7306ec1ce","Type":"ContainerStarted","Data":"eeebccf13fef7dce1dd5696e0c5eaa187cc30b866b9922d18c02f6220e095802"} Sep 15 12:30:01.413387 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:01.412995 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm" event={"ID":"62512f37-af98-4011-9d99-fbe7306ec1ce","Type":"ContainerStarted","Data":"68551121ce6aaabcfad2c301948e4d328656d637c01e8289d721711089c463e8"} Sep 15 12:30:01.414478 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:01.414453 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq" event={"ID":"77a7581f-3035-4803-881d-a884a9a1111d","Type":"ContainerStarted","Data":"07a19a8c780f531b3346553dd1009b2b09280fa4e5a7bb0d0360831059d83fc0"} Sep 15 12:30:01.414621 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:01.414523 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq" event={"ID":"77a7581f-3035-4803-881d-a884a9a1111d","Type":"ContainerStarted","Data":"6c6093440bcc8d3baedc423cb9f19a844fdfc254629859e21702f285400e39c2"} Sep 15 12:30:01.428092 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:01.428035 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm" podStartSLOduration=1.428013361 podStartE2EDuration="1.428013361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:30:01.42798412 +0000 UTC m=+1653.185089192" watchObservedRunningTime="2026-09-15 12:30:01.428013361 +0000 UTC m=+1653.185118447" Sep 15 12:30:01.442726 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:01.442661 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq" podStartSLOduration=1.442646075 podStartE2EDuration="1.442646075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:30:01.441045947 +0000 UTC m=+1653.198151018" watchObservedRunningTime="2026-09-15 12:30:01.442646075 +0000 UTC m=+1653.199751145" Sep 15 12:30:02.422143 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:02.422104 2582 generic.go:356] "Generic (PLEG): container finished" podID="62512f37-af98-4011-9d99-fbe7306ec1ce" containerID="eeebccf13fef7dce1dd5696e0c5eaa187cc30b866b9922d18c02f6220e095802" exitCode=0 Sep 15 12:30:02.422596 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:02.422195 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm" event={"ID":"62512f37-af98-4011-9d99-fbe7306ec1ce","Type":"ContainerDied","Data":"eeebccf13fef7dce1dd5696e0c5eaa187cc30b866b9922d18c02f6220e095802"} Sep 15 12:30:02.423728 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:02.423709 2582 generic.go:356] "Generic (PLEG): container finished" podID="77a7581f-3035-4803-881d-a884a9a1111d" containerID="07a19a8c780f531b3346553dd1009b2b09280fa4e5a7bb0d0360831059d83fc0" exitCode=0 Sep 15 12:30:02.423813 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:02.423793 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq" event={"ID":"77a7581f-3035-4803-881d-a884a9a1111d","Type":"ContainerDied","Data":"07a19a8c780f531b3346553dd1009b2b09280fa4e5a7bb0d0360831059d83fc0"} Sep 15 12:30:03.582907 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:03.582872 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq" Sep 15 12:30:03.585635 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:03.585611 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm" Sep 15 12:30:04.434961 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:04.434926 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm" Sep 15 12:30:04.435147 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:04.434957 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824590-v89wm" event={"ID":"62512f37-af98-4011-9d99-fbe7306ec1ce","Type":"ContainerDied","Data":"68551121ce6aaabcfad2c301948e4d328656d637c01e8289d721711089c463e8"} Sep 15 12:30:04.435147 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:04.435006 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68551121ce6aaabcfad2c301948e4d328656d637c01e8289d721711089c463e8" Sep 15 12:30:04.436273 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:04.436253 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq" Sep 15 12:30:04.436387 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:04.436283 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq" event={"ID":"77a7581f-3035-4803-881d-a884a9a1111d","Type":"ContainerDied","Data":"6c6093440bcc8d3baedc423cb9f19a844fdfc254629859e21702f285400e39c2"} Sep 15 12:30:04.436387 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:04.436324 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c6093440bcc8d3baedc423cb9f19a844fdfc254629859e21702f285400e39c2" Sep 15 12:30:28.992079 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:30:28.992043 2582 scope.go:117] "RemoveContainer" containerID="70453d617970433943125696ad20f6dceff1aaf3a5f491a3541eecf5cac06e19" Sep 15 12:32:18.884595 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.884559 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj"] Sep 15 12:32:18.887015 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.884971 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62512f37-af98-4011-9d99-fbe7306ec1ce" containerName="cleanup" Sep 15 12:32:18.887015 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.884983 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="62512f37-af98-4011-9d99-fbe7306ec1ce" containerName="cleanup" Sep 15 12:32:18.887015 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.884994 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a7581f-3035-4803-881d-a884a9a1111d" containerName="cleanup" Sep 15 12:32:18.887015 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.884999 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a7581f-3035-4803-881d-a884a9a1111d" containerName="cleanup" Sep 15 12:32:18.887015 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.885064 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="62512f37-af98-4011-9d99-fbe7306ec1ce" containerName="cleanup" Sep 15 12:32:18.887015 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.885074 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="77a7581f-3035-4803-881d-a884a9a1111d" containerName="cleanup" Sep 15 12:32:18.888265 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.888248 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:32:18.891060 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.891036 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-ed21e2-dockercfg-rmjts\"" Sep 15 12:32:18.891158 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.891081 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-ed21e2\"" Sep 15 12:32:18.899277 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.899253 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj"] Sep 15 12:32:18.917081 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.917044 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb"] Sep 15 12:32:18.921456 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.921432 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:32:18.930292 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.930263 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb"] Sep 15 12:32:18.989017 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.988980 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c61e74c8-a235-4244-a024-a8480de92824-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb\" (UID: \"c61e74c8-a235-4244-a024-a8480de92824\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:32:18.989199 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.989033 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh7kf\" (UniqueName: \"kubernetes.io/projected/c61e74c8-a235-4244-a024-a8480de92824-kube-api-access-bh7kf\") pod \"payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb\" (UID: \"c61e74c8-a235-4244-a024-a8480de92824\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:32:18.989199 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.989140 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35e56da5-cab1-4866-b43f-bb13b23a68d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj\" (UID: \"35e56da5-cab1-4866-b43f-bb13b23a68d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:32:18.989294 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:18.989199 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wdr6\" (UniqueName: \"kubernetes.io/projected/35e56da5-cab1-4866-b43f-bb13b23a68d0-kube-api-access-9wdr6\") pod \"payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj\" (UID: \"35e56da5-cab1-4866-b43f-bb13b23a68d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:32:19.090761 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.090725 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35e56da5-cab1-4866-b43f-bb13b23a68d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj\" (UID: \"35e56da5-cab1-4866-b43f-bb13b23a68d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:32:19.090953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.090787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wdr6\" (UniqueName: \"kubernetes.io/projected/35e56da5-cab1-4866-b43f-bb13b23a68d0-kube-api-access-9wdr6\") pod \"payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj\" (UID: \"35e56da5-cab1-4866-b43f-bb13b23a68d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:32:19.090953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.090832 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c61e74c8-a235-4244-a024-a8480de92824-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb\" (UID: \"c61e74c8-a235-4244-a024-a8480de92824\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:32:19.090953 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.090852 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh7kf\" (UniqueName: \"kubernetes.io/projected/c61e74c8-a235-4244-a024-a8480de92824-kube-api-access-bh7kf\") pod \"payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb\" (UID: \"c61e74c8-a235-4244-a024-a8480de92824\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:32:19.091704 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.091680 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35e56da5-cab1-4866-b43f-bb13b23a68d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj\" (UID: \"35e56da5-cab1-4866-b43f-bb13b23a68d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:32:19.091770 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.091680 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c61e74c8-a235-4244-a024-a8480de92824-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb\" (UID: \"c61e74c8-a235-4244-a024-a8480de92824\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:32:19.100375 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.100349 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wdr6\" (UniqueName: \"kubernetes.io/projected/35e56da5-cab1-4866-b43f-bb13b23a68d0-kube-api-access-9wdr6\") pod \"payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj\" (UID: \"35e56da5-cab1-4866-b43f-bb13b23a68d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:32:19.100521 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.100381 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh7kf\" (UniqueName: \"kubernetes.io/projected/c61e74c8-a235-4244-a024-a8480de92824-kube-api-access-bh7kf\") pod \"payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb\" (UID: \"c61e74c8-a235-4244-a024-a8480de92824\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:32:19.199100 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.199000 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:32:19.233377 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.233344 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:32:19.476882 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.476856 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj"] Sep 15 12:32:19.478948 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:32:19.478920 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e56da5_cab1_4866_b43f_bb13b23a68d0.slice/crio-dad9d7d3bf161b0ec566a4c606e64f48c3a5386f8b66314dd31000e1d2d0f8e4 WatchSource:0}: Error finding container dad9d7d3bf161b0ec566a4c606e64f48c3a5386f8b66314dd31000e1d2d0f8e4: Status 404 returned error can't find the container with id dad9d7d3bf161b0ec566a4c606e64f48c3a5386f8b66314dd31000e1d2d0f8e4 Sep 15 12:32:19.521672 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:19.521645 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb"] Sep 15 12:32:19.525278 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:32:19.525249 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61e74c8_a235_4244_a024_a8480de92824.slice/crio-a2ec502531c169cf38a238867855c91b3fac3693a2422788260fc01a836c7d0f WatchSource:0}: Error finding container a2ec502531c169cf38a238867855c91b3fac3693a2422788260fc01a836c7d0f: Status 404 returned error can't find the container with id a2ec502531c169cf38a238867855c91b3fac3693a2422788260fc01a836c7d0f Sep 15 12:32:20.138349 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:20.138310 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" event={"ID":"c61e74c8-a235-4244-a024-a8480de92824","Type":"ContainerStarted","Data":"ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934"} Sep 15 12:32:20.138861 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:20.138376 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" event={"ID":"c61e74c8-a235-4244-a024-a8480de92824","Type":"ContainerStarted","Data":"a2ec502531c169cf38a238867855c91b3fac3693a2422788260fc01a836c7d0f"} Sep 15 12:32:20.138861 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:20.138599 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:32:20.140000 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:20.139975 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" event={"ID":"35e56da5-cab1-4866-b43f-bb13b23a68d0","Type":"ContainerStarted","Data":"1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4"} Sep 15 12:32:20.140137 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:20.140018 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" event={"ID":"35e56da5-cab1-4866-b43f-bb13b23a68d0","Type":"ContainerStarted","Data":"dad9d7d3bf161b0ec566a4c606e64f48c3a5386f8b66314dd31000e1d2d0f8e4"} Sep 15 12:32:20.140137 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:20.140075 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:32:20.155101 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:20.155046 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" podStartSLOduration=2.155025828 podStartE2EDuration="2.155025828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:32:20.154611827 +0000 UTC m=+1791.911716899" watchObservedRunningTime="2026-09-15 12:32:20.155025828 +0000 UTC m=+1791.912130900" Sep 15 12:32:20.171221 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:20.171170 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" podStartSLOduration=2.171153542 podStartE2EDuration="2.171153542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:32:20.169941207 +0000 UTC m=+1791.927046270" watchObservedRunningTime="2026-09-15 12:32:20.171153542 +0000 UTC m=+1791.928258613" Sep 15 12:32:31.148190 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:31.148159 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:32:31.148604 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:31.148230 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:32:49.289967 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.289932 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5"] Sep 15 12:32:49.294112 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.294079 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:32:49.298758 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.298732 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-7b204e-dockercfg-gqgfh\"" Sep 15 12:32:49.298919 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.298735 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-7b204e\"" Sep 15 12:32:49.304125 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.303777 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5"] Sep 15 12:32:49.393232 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.393190 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzd7c\" (UniqueName: \"kubernetes.io/projected/181123e6-283f-481d-af02-0ca0c983638e-kube-api-access-mzd7c\") pod \"payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5\" (UID: \"181123e6-283f-481d-af02-0ca0c983638e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:32:49.393451 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.393254 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/181123e6-283f-481d-af02-0ca0c983638e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5\" (UID: \"181123e6-283f-481d-af02-0ca0c983638e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:32:49.494330 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.494287 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzd7c\" (UniqueName: \"kubernetes.io/projected/181123e6-283f-481d-af02-0ca0c983638e-kube-api-access-mzd7c\") pod \"payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5\" (UID: \"181123e6-283f-481d-af02-0ca0c983638e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:32:49.494599 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.494342 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/181123e6-283f-481d-af02-0ca0c983638e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5\" (UID: \"181123e6-283f-481d-af02-0ca0c983638e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:32:49.495068 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.495048 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/181123e6-283f-481d-af02-0ca0c983638e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5\" (UID: \"181123e6-283f-481d-af02-0ca0c983638e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:32:49.502510 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.502471 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzd7c\" (UniqueName: \"kubernetes.io/projected/181123e6-283f-481d-af02-0ca0c983638e-kube-api-access-mzd7c\") pod \"payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5\" (UID: \"181123e6-283f-481d-af02-0ca0c983638e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:32:49.609529 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.609402 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:32:49.884121 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:32:49.884091 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod181123e6_283f_481d_af02_0ca0c983638e.slice/crio-3a0ecb7aad3e2dc807c018273a49c61b9bc51819e18627006c035f3fefb51d6e WatchSource:0}: Error finding container 3a0ecb7aad3e2dc807c018273a49c61b9bc51819e18627006c035f3fefb51d6e: Status 404 returned error can't find the container with id 3a0ecb7aad3e2dc807c018273a49c61b9bc51819e18627006c035f3fefb51d6e Sep 15 12:32:49.885276 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:49.885251 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5"] Sep 15 12:32:50.303126 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:50.303090 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" event={"ID":"181123e6-283f-481d-af02-0ca0c983638e","Type":"ContainerStarted","Data":"e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208"} Sep 15 12:32:50.303606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:50.303155 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:32:50.303606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:50.303174 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" event={"ID":"181123e6-283f-481d-af02-0ca0c983638e","Type":"ContainerStarted","Data":"3a0ecb7aad3e2dc807c018273a49c61b9bc51819e18627006c035f3fefb51d6e"} Sep 15 12:32:50.320705 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:32:50.320645 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" podStartSLOduration=1.320627246 podStartE2EDuration="1.320627246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:32:50.318965496 +0000 UTC m=+1822.076070569" watchObservedRunningTime="2026-09-15 12:32:50.320627246 +0000 UTC m=+1822.077732317" Sep 15 12:33:01.310715 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:01.310676 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:33:16.182622 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:16.182585 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-clfp7_bd363dc1-4a24-4aed-83d0-d5bcbdedd04f/payload-processing/0.log" Sep 15 12:33:16.308811 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:16.308784 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5_181123e6-283f-481d-af02-0ca0c983638e/payload-processing/0.log" Sep 15 12:33:57.370422 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:57.370392 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb_c61e74c8-a235-4244-a024-a8480de92824/payload-processing/0.log" Sep 15 12:33:57.496446 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:57.496417 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-clfp7_bd363dc1-4a24-4aed-83d0-d5bcbdedd04f/payload-processing/0.log" Sep 15 12:33:57.617265 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:57.617234 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5_181123e6-283f-481d-af02-0ca0c983638e/payload-processing/0.log" Sep 15 12:33:58.180597 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.180564 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt"] Sep 15 12:33:58.184411 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.184386 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.187018 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.186992 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-9725ba-openshift-default-dockercfg-j2h2d\"" Sep 15 12:33:58.199462 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.199423 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt"] Sep 15 12:33:58.272493 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.272455 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-socket\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.272692 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.272514 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-envoy\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.272692 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.272584 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-token\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.272692 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.272669 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.272852 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.272746 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-data\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.272852 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.272815 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scxz6\" (UniqueName: \"kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-kube-api-access-scxz6\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.272940 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.272885 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-certs\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.272981 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.272938 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-credential-socket\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.273033 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.272992 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-podinfo\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.374425 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.374381 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scxz6\" (UniqueName: \"kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-kube-api-access-scxz6\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.374898 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.374434 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-certs\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.374898 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.374473 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-credential-socket\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.374898 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.374555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-podinfo\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.374898 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.374601 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-socket\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.374898 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.374623 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-envoy\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.374898 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.374645 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-token\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.374898 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.374672 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.374898 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.374727 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-data\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.375345 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.374984 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-certs\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.375345 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.375017 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-credential-socket\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.375345 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.375102 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-data\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.375696 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.375672 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-socket\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.375799 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.375670 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.377175 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.377156 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-envoy\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.377583 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.377561 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-podinfo\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.382299 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.382280 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-token\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.382688 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.382665 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scxz6\" (UniqueName: \"kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-kube-api-access-scxz6\") pod \"e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.503171 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.503128 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:33:58.789251 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.789221 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt"] Sep 15 12:33:58.791128 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:33:58.791100 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbec5ae9_57ec_41cd_85e0_0b11bc240e1d.slice/crio-d2ae0ea8fdb74f639f9b6f91ebbc37d04983fdc6445e747fa08231cb231996a5 WatchSource:0}: Error finding container d2ae0ea8fdb74f639f9b6f91ebbc37d04983fdc6445e747fa08231cb231996a5: Status 404 returned error can't find the container with id d2ae0ea8fdb74f639f9b6f91ebbc37d04983fdc6445e747fa08231cb231996a5 Sep 15 12:33:58.793521 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.793462 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:33:58.793629 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.793566 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:33:58.793629 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:58.793601 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:33:59.664916 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:59.664874 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" event={"ID":"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d","Type":"ContainerStarted","Data":"5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6"} Sep 15 12:33:59.665323 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:59.664943 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" event={"ID":"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d","Type":"ContainerStarted","Data":"d2ae0ea8fdb74f639f9b6f91ebbc37d04983fdc6445e747fa08231cb231996a5"} Sep 15 12:33:59.686664 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:33:59.686587 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" podStartSLOduration=1.686570619 podStartE2EDuration="1.686570619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:33:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:33:59.685276765 +0000 UTC m=+1891.442381836" watchObservedRunningTime="2026-09-15 12:33:59.686570619 +0000 UTC m=+1891.443675690" Sep 15 12:34:00.504020 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:00.503973 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:34:00.510171 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:00.510132 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:34:00.669947 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:00.669920 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:34:00.671501 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:00.671463 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:34:05.727667 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.727625 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm"] Sep 15 12:34:05.731079 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.731062 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:05.733768 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.733744 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-9725ba\"" Sep 15 12:34:05.734004 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.733988 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-9725ba-dockercfg-7r648\"" Sep 15 12:34:05.739789 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.739768 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm"] Sep 15 12:34:05.752065 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.752028 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9qbv\" (UniqueName: \"kubernetes.io/projected/7a71da45-1f7d-40de-8b47-91706be0d77d-kube-api-access-s9qbv\") pod \"payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm\" (UID: \"7a71da45-1f7d-40de-8b47-91706be0d77d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:05.752237 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.752173 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a71da45-1f7d-40de-8b47-91706be0d77d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm\" (UID: \"7a71da45-1f7d-40de-8b47-91706be0d77d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:05.853175 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.853144 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a71da45-1f7d-40de-8b47-91706be0d77d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm\" (UID: \"7a71da45-1f7d-40de-8b47-91706be0d77d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:05.853350 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.853243 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9qbv\" (UniqueName: \"kubernetes.io/projected/7a71da45-1f7d-40de-8b47-91706be0d77d-kube-api-access-s9qbv\") pod \"payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm\" (UID: \"7a71da45-1f7d-40de-8b47-91706be0d77d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:05.853832 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.853810 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a71da45-1f7d-40de-8b47-91706be0d77d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm\" (UID: \"7a71da45-1f7d-40de-8b47-91706be0d77d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:05.861593 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:05.861568 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9qbv\" (UniqueName: \"kubernetes.io/projected/7a71da45-1f7d-40de-8b47-91706be0d77d-kube-api-access-s9qbv\") pod \"payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm\" (UID: \"7a71da45-1f7d-40de-8b47-91706be0d77d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:06.043015 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:06.042919 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:06.315986 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:06.315958 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm"] Sep 15 12:34:06.318037 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:34:06.318005 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a71da45_1f7d_40de_8b47_91706be0d77d.slice/crio-29256c1b23d854badb6ffd79d290b58e74e41cb11a08364e75c15c023542e72e WatchSource:0}: Error finding container 29256c1b23d854badb6ffd79d290b58e74e41cb11a08364e75c15c023542e72e: Status 404 returned error can't find the container with id 29256c1b23d854badb6ffd79d290b58e74e41cb11a08364e75c15c023542e72e Sep 15 12:34:06.712173 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:06.712091 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:06.712173 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:06.712138 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" event={"ID":"7a71da45-1f7d-40de-8b47-91706be0d77d","Type":"ContainerStarted","Data":"754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d"} Sep 15 12:34:06.712173 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:06.712164 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" event={"ID":"7a71da45-1f7d-40de-8b47-91706be0d77d","Type":"ContainerStarted","Data":"29256c1b23d854badb6ffd79d290b58e74e41cb11a08364e75c15c023542e72e"} Sep 15 12:34:06.726748 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:06.726682 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" podStartSLOduration=1.726659891 podStartE2EDuration="1.726659891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:34:06.725452978 +0000 UTC m=+1898.482558050" watchObservedRunningTime="2026-09-15 12:34:06.726659891 +0000 UTC m=+1898.483764967" Sep 15 12:34:17.715447 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:17.715407 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:20.781370 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:20.781285 2582 generic.go:356] "Generic (PLEG): container finished" podID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerID="5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6" exitCode=0 Sep 15 12:34:20.781804 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:20.781366 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" event={"ID":"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d","Type":"ContainerDied","Data":"5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6"} Sep 15 12:34:20.781847 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:20.781818 2582 scope.go:117] "RemoveContainer" containerID="5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6" Sep 15 12:34:20.783039 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:20.783012 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:34:20.783101 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:20.783080 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:34:20.783140 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:20.783110 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 15 12:34:21.789008 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:21.788971 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" event={"ID":"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d","Type":"ContainerStarted","Data":"8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80"} Sep 15 12:34:22.503376 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:22.503325 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:34:22.503787 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:22.503756 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Sep 15 12:34:22.503890 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:22.503816 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Sep 15 12:34:23.503797 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:23.503742 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Sep 15 12:34:23.504323 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:23.503820 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Sep 15 12:34:24.504669 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:24.504615 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Sep 15 12:34:24.505200 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:24.504730 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Sep 15 12:34:25.504397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:25.503888 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Sep 15 12:34:25.504397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:25.503963 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Sep 15 12:34:26.503864 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:26.503825 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Sep 15 12:34:26.504947 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:26.504452 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Sep 15 12:34:27.536266 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:27.536233 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:34:27.536763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:27.536737 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:34:27.537584 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:27.537564 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:34:43.356232 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.356189 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm"] Sep 15 12:34:43.356874 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.356467 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" podUID="7a71da45-1f7d-40de-8b47-91706be0d77d" containerName="payload-processing" containerID="cri-o://754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d" gracePeriod=30 Sep 15 12:34:43.622841 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.622812 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:43.740201 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.740161 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a71da45-1f7d-40de-8b47-91706be0d77d-plugins-config-volume\") pod \"7a71da45-1f7d-40de-8b47-91706be0d77d\" (UID: \"7a71da45-1f7d-40de-8b47-91706be0d77d\") " Sep 15 12:34:43.740403 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.740249 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9qbv\" (UniqueName: \"kubernetes.io/projected/7a71da45-1f7d-40de-8b47-91706be0d77d-kube-api-access-s9qbv\") pod \"7a71da45-1f7d-40de-8b47-91706be0d77d\" (UID: \"7a71da45-1f7d-40de-8b47-91706be0d77d\") " Sep 15 12:34:43.740615 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.740589 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a71da45-1f7d-40de-8b47-91706be0d77d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a71da45-1f7d-40de-8b47-91706be0d77d" (UID: "7a71da45-1f7d-40de-8b47-91706be0d77d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:34:43.740716 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.740701 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a71da45-1f7d-40de-8b47-91706be0d77d-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:43.742449 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.742426 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a71da45-1f7d-40de-8b47-91706be0d77d-kube-api-access-s9qbv" (OuterVolumeSpecName: "kube-api-access-s9qbv") pod "7a71da45-1f7d-40de-8b47-91706be0d77d" (UID: "7a71da45-1f7d-40de-8b47-91706be0d77d"). InnerVolumeSpecName "kube-api-access-s9qbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:34:43.841713 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.841675 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9qbv\" (UniqueName: \"kubernetes.io/projected/7a71da45-1f7d-40de-8b47-91706be0d77d-kube-api-access-s9qbv\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:43.922586 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.922469 2582 generic.go:356] "Generic (PLEG): container finished" podID="7a71da45-1f7d-40de-8b47-91706be0d77d" containerID="754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d" exitCode=0 Sep 15 12:34:43.922586 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.922531 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" Sep 15 12:34:43.922586 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.922560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" event={"ID":"7a71da45-1f7d-40de-8b47-91706be0d77d","Type":"ContainerDied","Data":"754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d"} Sep 15 12:34:43.922894 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.922624 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm" event={"ID":"7a71da45-1f7d-40de-8b47-91706be0d77d","Type":"ContainerDied","Data":"29256c1b23d854badb6ffd79d290b58e74e41cb11a08364e75c15c023542e72e"} Sep 15 12:34:43.922894 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.922659 2582 scope.go:117] "RemoveContainer" containerID="754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d" Sep 15 12:34:43.932551 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.932528 2582 scope.go:117] "RemoveContainer" containerID="754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d" Sep 15 12:34:43.932870 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:34:43.932852 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d\": container with ID starting with 754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d not found: ID does not exist" containerID="754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d" Sep 15 12:34:43.932929 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.932880 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d"} err="failed to get container status \"754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d\": rpc error: code = NotFound desc = could not find container \"754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d\": container with ID starting with 754597a5b7a0e4ee238de0c19929d58e8b912eac9404aed1900900d511ee605d not found: ID does not exist" Sep 15 12:34:43.945475 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.945446 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm"] Sep 15 12:34:43.949333 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:43.949305 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9725ba-588f7dd55-cdzbm"] Sep 15 12:34:44.712372 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:44.712335 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a71da45-1f7d-40de-8b47-91706be0d77d" path="/var/lib/kubelet/pods/7a71da45-1f7d-40de-8b47-91706be0d77d/volumes" Sep 15 12:34:50.262289 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:50.262254 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt"] Sep 15 12:34:50.262710 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:50.262540 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" containerID="cri-o://8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80" gracePeriod=30 Sep 15 12:34:54.120519 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.120464 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh"] Sep 15 12:34:54.120935 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.120897 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a71da45-1f7d-40de-8b47-91706be0d77d" containerName="payload-processing" Sep 15 12:34:54.120935 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.120912 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a71da45-1f7d-40de-8b47-91706be0d77d" containerName="payload-processing" Sep 15 12:34:54.121040 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.120984 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a71da45-1f7d-40de-8b47-91706be0d77d" containerName="payload-processing" Sep 15 12:34:54.124571 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.124547 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh" Sep 15 12:34:54.127119 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.127096 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:34:54.127237 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.127098 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:34:54.136681 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.136382 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh"] Sep 15 12:34:54.238985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.238936 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf599e3-990d-4694-b342-20c86654fb1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh\" (UID: \"9cf599e3-990d-4694-b342-20c86654fb1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh" Sep 15 12:34:54.339985 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.339942 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf599e3-990d-4694-b342-20c86654fb1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh\" (UID: \"9cf599e3-990d-4694-b342-20c86654fb1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh" Sep 15 12:34:54.340606 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.340587 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf599e3-990d-4694-b342-20c86654fb1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh\" (UID: \"9cf599e3-990d-4694-b342-20c86654fb1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh" Sep 15 12:34:54.436765 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.436668 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh" Sep 15 12:34:54.716348 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.716323 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh"] Sep 15 12:34:54.718391 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:34:54.718366 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf599e3_990d_4694_b342_20c86654fb1b.slice/crio-604cafcf57da23c002ed21d36bf5febee9f5ec5bf1bb13f39936e85384c9fa31 WatchSource:0}: Error finding container 604cafcf57da23c002ed21d36bf5febee9f5ec5bf1bb13f39936e85384c9fa31: Status 404 returned error can't find the container with id 604cafcf57da23c002ed21d36bf5febee9f5ec5bf1bb13f39936e85384c9fa31 Sep 15 12:34:54.986613 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:54.986523 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh" event={"ID":"9cf599e3-990d-4694-b342-20c86654fb1b","Type":"ContainerStarted","Data":"604cafcf57da23c002ed21d36bf5febee9f5ec5bf1bb13f39936e85384c9fa31"} Sep 15 12:34:55.528678 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.528606 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:34:55.653175 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653140 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istiod-ca-cert\") pod \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " Sep 15 12:34:55.653364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653186 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-podinfo\") pod \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " Sep 15 12:34:55.653364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653222 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scxz6\" (UniqueName: \"kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-kube-api-access-scxz6\") pod \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " Sep 15 12:34:55.653364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653291 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-certs\") pod \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " Sep 15 12:34:55.653364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653339 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-envoy\") pod \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " Sep 15 12:34:55.653364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653363 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-socket\") pod \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " Sep 15 12:34:55.653673 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653412 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-token\") pod \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " Sep 15 12:34:55.653673 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653442 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-data\") pod \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " Sep 15 12:34:55.653673 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653475 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-credential-socket\") pod \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\" (UID: \"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d\") " Sep 15 12:34:55.653673 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653550 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" (UID: "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:34:55.653885 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653716 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" (UID: "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:34:55.653885 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653799 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" (UID: "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:34:55.653885 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653833 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-data" (OuterVolumeSpecName: "istio-data") pod "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" (UID: "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:34:55.654048 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.653981 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" (UID: "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:34:55.654313 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.654297 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istiod-ca-cert\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:55.654401 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.654317 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-certs\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:55.654401 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.654330 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-workload-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:55.654401 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.654343 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-data\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:55.654401 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.654357 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-credential-socket\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:55.655555 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.655530 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" (UID: "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 12:34:55.655964 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.655942 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-token" (OuterVolumeSpecName: "istio-token") pod "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" (UID: "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:34:55.656157 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.656139 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-kube-api-access-scxz6" (OuterVolumeSpecName: "kube-api-access-scxz6") pod "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" (UID: "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d"). InnerVolumeSpecName "kube-api-access-scxz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:34:55.656214 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.656149 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" (UID: "bbec5ae9-57ec-41cd-85e0-0b11bc240e1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 12:34:55.755314 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.755279 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-envoy\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:55.755314 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.755307 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-token\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:55.755314 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.755317 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-istio-podinfo\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:55.755613 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.755325 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scxz6\" (UniqueName: \"kubernetes.io/projected/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d-kube-api-access-scxz6\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:55.996317 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.994603 2582 generic.go:356] "Generic (PLEG): container finished" podID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerID="8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80" exitCode=0 Sep 15 12:34:55.996317 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.994797 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" event={"ID":"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d","Type":"ContainerDied","Data":"8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80"} Sep 15 12:34:55.996317 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.994908 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" event={"ID":"bbec5ae9-57ec-41cd-85e0-0b11bc240e1d","Type":"ContainerDied","Data":"d2ae0ea8fdb74f639f9b6f91ebbc37d04983fdc6445e747fa08231cb231996a5"} Sep 15 12:34:55.996317 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.994969 2582 scope.go:117] "RemoveContainer" containerID="8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80" Sep 15 12:34:55.996317 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:55.995239 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt" Sep 15 12:34:56.004369 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:56.004338 2582 generic.go:356] "Generic (PLEG): container finished" podID="9cf599e3-990d-4694-b342-20c86654fb1b" containerID="5b423d11f3d22430b8099938a521a10073d642a41d1ddf9403f38b3314550220" exitCode=0 Sep 15 12:34:56.004538 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:56.004387 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh" event={"ID":"9cf599e3-990d-4694-b342-20c86654fb1b","Type":"ContainerDied","Data":"5b423d11f3d22430b8099938a521a10073d642a41d1ddf9403f38b3314550220"} Sep 15 12:34:56.012254 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:56.012233 2582 scope.go:117] "RemoveContainer" containerID="5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6" Sep 15 12:34:56.021904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:56.021886 2582 scope.go:117] "RemoveContainer" containerID="8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80" Sep 15 12:34:56.022223 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:34:56.022200 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80\": container with ID starting with 8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80 not found: ID does not exist" containerID="8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80" Sep 15 12:34:56.022319 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:56.022235 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80"} err="failed to get container status \"8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80\": rpc error: code = NotFound desc = could not find container \"8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80\": container with ID starting with 8c9eda6fd5bc075ea83cf011641a3f7a7f73dae79d62697d7712ae81d7f9ae80 not found: ID does not exist" Sep 15 12:34:56.022319 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:56.022258 2582 scope.go:117] "RemoveContainer" containerID="5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6" Sep 15 12:34:56.022545 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:34:56.022525 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6\": container with ID starting with 5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6 not found: ID does not exist" containerID="5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6" Sep 15 12:34:56.022601 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:56.022553 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6"} err="failed to get container status \"5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6\": rpc error: code = NotFound desc = could not find container \"5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6\": container with ID starting with 5e9366904020990d2b273d2483b60e335e7b9e7aad3ad36fb7f4ea4d84d072d6 not found: ID does not exist" Sep 15 12:34:56.032417 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:56.032393 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt"] Sep 15 12:34:56.036112 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:56.036089 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9725ba-openshift-default-69bf78d668-sv6zt"] Sep 15 12:34:56.712590 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:56.712556 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" path="/var/lib/kubelet/pods/bbec5ae9-57ec-41cd-85e0-0b11bc240e1d/volumes" Sep 15 12:34:57.150851 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:57.150826 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh" Sep 15 12:34:57.270363 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:57.270026 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf599e3-990d-4694-b342-20c86654fb1b-maas-api-service-ca\") pod \"9cf599e3-990d-4694-b342-20c86654fb1b\" (UID: \"9cf599e3-990d-4694-b342-20c86654fb1b\") " Sep 15 12:34:57.270742 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:57.270647 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf599e3-990d-4694-b342-20c86654fb1b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9cf599e3-990d-4694-b342-20c86654fb1b" (UID: "9cf599e3-990d-4694-b342-20c86654fb1b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:34:57.371796 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:57.371748 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf599e3-990d-4694-b342-20c86654fb1b-maas-api-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:34:58.018532 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:58.018500 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh" Sep 15 12:34:58.018532 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:58.018478 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh" event={"ID":"9cf599e3-990d-4694-b342-20c86654fb1b","Type":"ContainerDied","Data":"604cafcf57da23c002ed21d36bf5febee9f5ec5bf1bb13f39936e85384c9fa31"} Sep 15 12:34:58.019027 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:34:58.018554 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="604cafcf57da23c002ed21d36bf5febee9f5ec5bf1bb13f39936e85384c9fa31" Sep 15 12:35:09.770725 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:09.770688 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb"] Sep 15 12:35:09.771216 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:09.770949 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" podUID="c61e74c8-a235-4244-a024-a8480de92824" containerName="payload-processing" containerID="cri-o://ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934" gracePeriod=30 Sep 15 12:35:09.775769 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:09.775669 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj"] Sep 15 12:35:09.775934 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:09.775889 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" podUID="35e56da5-cab1-4866-b43f-bb13b23a68d0" containerName="payload-pre-processing" containerID="cri-o://1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4" gracePeriod=30 Sep 15 12:35:09.940580 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:09.940544 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh"] Sep 15 12:35:09.942904 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:09.942879 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ed21e2-l74sh"] Sep 15 12:35:10.079018 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.078632 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:35:10.083898 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.083869 2582 generic.go:356] "Generic (PLEG): container finished" podID="c61e74c8-a235-4244-a024-a8480de92824" containerID="ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934" exitCode=0 Sep 15 12:35:10.084049 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.083913 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" Sep 15 12:35:10.084049 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.083946 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" event={"ID":"c61e74c8-a235-4244-a024-a8480de92824","Type":"ContainerDied","Data":"ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934"} Sep 15 12:35:10.084049 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.083995 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb" event={"ID":"c61e74c8-a235-4244-a024-a8480de92824","Type":"ContainerDied","Data":"a2ec502531c169cf38a238867855c91b3fac3693a2422788260fc01a836c7d0f"} Sep 15 12:35:10.084049 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.084017 2582 scope.go:117] "RemoveContainer" containerID="ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934" Sep 15 12:35:10.084664 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.084627 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:35:10.085325 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.085306 2582 generic.go:356] "Generic (PLEG): container finished" podID="35e56da5-cab1-4866-b43f-bb13b23a68d0" containerID="1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4" exitCode=0 Sep 15 12:35:10.085419 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.085356 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" event={"ID":"35e56da5-cab1-4866-b43f-bb13b23a68d0","Type":"ContainerDied","Data":"1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4"} Sep 15 12:35:10.085419 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.085386 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" event={"ID":"35e56da5-cab1-4866-b43f-bb13b23a68d0","Type":"ContainerDied","Data":"dad9d7d3bf161b0ec566a4c606e64f48c3a5386f8b66314dd31000e1d2d0f8e4"} Sep 15 12:35:10.093320 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.093299 2582 scope.go:117] "RemoveContainer" containerID="ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934" Sep 15 12:35:10.093701 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:35:10.093678 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934\": container with ID starting with ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934 not found: ID does not exist" containerID="ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934" Sep 15 12:35:10.093812 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.093709 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934"} err="failed to get container status \"ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934\": rpc error: code = NotFound desc = could not find container \"ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934\": container with ID starting with ad622b501ccfcf2a38b5bf5ac63b40012e7b5a96f94024a76cef211866f7d934 not found: ID does not exist" Sep 15 12:35:10.093812 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.093729 2582 scope.go:117] "RemoveContainer" containerID="1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4" Sep 15 12:35:10.104370 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.104349 2582 scope.go:117] "RemoveContainer" containerID="1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4" Sep 15 12:35:10.104823 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:35:10.104799 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4\": container with ID starting with 1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4 not found: ID does not exist" containerID="1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4" Sep 15 12:35:10.104932 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.104833 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4"} err="failed to get container status \"1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4\": rpc error: code = NotFound desc = could not find container \"1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4\": container with ID starting with 1698ea83b6f682974e14ab93cc4e8e63440b46915c7b8d3664abcfbf1e1cdbf4 not found: ID does not exist" Sep 15 12:35:10.204395 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.204352 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35e56da5-cab1-4866-b43f-bb13b23a68d0-plugins-config-volume\") pod \"35e56da5-cab1-4866-b43f-bb13b23a68d0\" (UID: \"35e56da5-cab1-4866-b43f-bb13b23a68d0\") " Sep 15 12:35:10.204395 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.204398 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wdr6\" (UniqueName: \"kubernetes.io/projected/35e56da5-cab1-4866-b43f-bb13b23a68d0-kube-api-access-9wdr6\") pod \"35e56da5-cab1-4866-b43f-bb13b23a68d0\" (UID: \"35e56da5-cab1-4866-b43f-bb13b23a68d0\") " Sep 15 12:35:10.204666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.204437 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh7kf\" (UniqueName: \"kubernetes.io/projected/c61e74c8-a235-4244-a024-a8480de92824-kube-api-access-bh7kf\") pod \"c61e74c8-a235-4244-a024-a8480de92824\" (UID: \"c61e74c8-a235-4244-a024-a8480de92824\") " Sep 15 12:35:10.204666 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.204580 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c61e74c8-a235-4244-a024-a8480de92824-plugins-config-volume\") pod \"c61e74c8-a235-4244-a024-a8480de92824\" (UID: \"c61e74c8-a235-4244-a024-a8480de92824\") " Sep 15 12:35:10.204946 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.204917 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35e56da5-cab1-4866-b43f-bb13b23a68d0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35e56da5-cab1-4866-b43f-bb13b23a68d0" (UID: "35e56da5-cab1-4866-b43f-bb13b23a68d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:35:10.205006 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.204975 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c61e74c8-a235-4244-a024-a8480de92824-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c61e74c8-a235-4244-a024-a8480de92824" (UID: "c61e74c8-a235-4244-a024-a8480de92824"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:35:10.206952 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.206926 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c61e74c8-a235-4244-a024-a8480de92824-kube-api-access-bh7kf" (OuterVolumeSpecName: "kube-api-access-bh7kf") pod "c61e74c8-a235-4244-a024-a8480de92824" (UID: "c61e74c8-a235-4244-a024-a8480de92824"). InnerVolumeSpecName "kube-api-access-bh7kf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:35:10.206952 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.206938 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35e56da5-cab1-4866-b43f-bb13b23a68d0-kube-api-access-9wdr6" (OuterVolumeSpecName: "kube-api-access-9wdr6") pod "35e56da5-cab1-4866-b43f-bb13b23a68d0" (UID: "35e56da5-cab1-4866-b43f-bb13b23a68d0"). InnerVolumeSpecName "kube-api-access-9wdr6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:35:10.306201 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.306105 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c61e74c8-a235-4244-a024-a8480de92824-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:35:10.306201 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.306138 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35e56da5-cab1-4866-b43f-bb13b23a68d0-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:35:10.306201 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.306148 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wdr6\" (UniqueName: \"kubernetes.io/projected/35e56da5-cab1-4866-b43f-bb13b23a68d0-kube-api-access-9wdr6\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:35:10.306201 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.306157 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh7kf\" (UniqueName: \"kubernetes.io/projected/c61e74c8-a235-4244-a024-a8480de92824-kube-api-access-bh7kf\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:35:10.408474 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.408439 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb"] Sep 15 12:35:10.410366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.410343 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ed21e2-5cd7f7f98c-jjnkb"] Sep 15 12:35:10.712409 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.712313 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cf599e3-990d-4694-b342-20c86654fb1b" path="/var/lib/kubelet/pods/9cf599e3-990d-4694-b342-20c86654fb1b/volumes" Sep 15 12:35:10.712779 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:10.712763 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c61e74c8-a235-4244-a024-a8480de92824" path="/var/lib/kubelet/pods/c61e74c8-a235-4244-a024-a8480de92824/volumes" Sep 15 12:35:11.091955 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:11.091858 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj" Sep 15 12:35:11.109914 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:11.109879 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj"] Sep 15 12:35:11.111983 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:11.111962 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ed21e2-5fcbdd8d4b-7l2cj"] Sep 15 12:35:12.711753 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:12.711721 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35e56da5-cab1-4866-b43f-bb13b23a68d0" path="/var/lib/kubelet/pods/35e56da5-cab1-4866-b43f-bb13b23a68d0/volumes" Sep 15 12:35:29.044186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:29.044149 2582 scope.go:117] "RemoveContainer" containerID="9a2ee27f02eb28ea7196cb43509d7b488987cff40cd3b58e9b80f0ed08c62aa3" Sep 15 12:35:59.029564 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.029524 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82"] Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.029943 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.029955 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.029964 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf599e3-990d-4694-b342-20c86654fb1b" containerName="revoke-keys" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.029969 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf599e3-990d-4694-b342-20c86654fb1b" containerName="revoke-keys" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.029978 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35e56da5-cab1-4866-b43f-bb13b23a68d0" containerName="payload-pre-processing" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.029985 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="35e56da5-cab1-4866-b43f-bb13b23a68d0" containerName="payload-pre-processing" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.030008 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c61e74c8-a235-4244-a024-a8480de92824" containerName="payload-processing" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.030014 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c61e74c8-a235-4244-a024-a8480de92824" containerName="payload-processing" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.030071 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.030079 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf599e3-990d-4694-b342-20c86654fb1b" containerName="revoke-keys" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.030088 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.030094 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="35e56da5-cab1-4866-b43f-bb13b23a68d0" containerName="payload-pre-processing" Sep 15 12:35:59.030132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.030099 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c61e74c8-a235-4244-a024-a8480de92824" containerName="payload-processing" Sep 15 12:35:59.033502 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.033451 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82"] Sep 15 12:35:59.033637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.033599 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82" Sep 15 12:35:59.036169 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.036142 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:35:59.036387 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.036172 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:35:59.160275 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.160238 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71510952-7fbd-45c5-99bb-75aff4a4bb94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82\" (UID: \"71510952-7fbd-45c5-99bb-75aff4a4bb94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82" Sep 15 12:35:59.262186 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.261559 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71510952-7fbd-45c5-99bb-75aff4a4bb94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82\" (UID: \"71510952-7fbd-45c5-99bb-75aff4a4bb94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82" Sep 15 12:35:59.262505 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.262455 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71510952-7fbd-45c5-99bb-75aff4a4bb94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82\" (UID: \"71510952-7fbd-45c5-99bb-75aff4a4bb94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82" Sep 15 12:35:59.347981 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.347881 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82" Sep 15 12:35:59.622609 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:35:59.622584 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82"] Sep 15 12:35:59.624039 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:35:59.624014 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71510952_7fbd_45c5_99bb_75aff4a4bb94.slice/crio-a4a106e5de928884d8bd78f5ce3e2dedd1ea99b225901b4a971f45c5f917b6c9 WatchSource:0}: Error finding container a4a106e5de928884d8bd78f5ce3e2dedd1ea99b225901b4a971f45c5f917b6c9: Status 404 returned error can't find the container with id a4a106e5de928884d8bd78f5ce3e2dedd1ea99b225901b4a971f45c5f917b6c9 Sep 15 12:36:00.344077 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:00.344043 2582 generic.go:356] "Generic (PLEG): container finished" podID="71510952-7fbd-45c5-99bb-75aff4a4bb94" containerID="ce68a92fd4c1d1f467d66d57db8c648e7e369d2dbc0b3bb3910a7fe0513b44d0" exitCode=0 Sep 15 12:36:00.344469 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:00.344130 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82" event={"ID":"71510952-7fbd-45c5-99bb-75aff4a4bb94","Type":"ContainerDied","Data":"ce68a92fd4c1d1f467d66d57db8c648e7e369d2dbc0b3bb3910a7fe0513b44d0"} Sep 15 12:36:00.344469 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:00.344187 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82" event={"ID":"71510952-7fbd-45c5-99bb-75aff4a4bb94","Type":"ContainerStarted","Data":"a4a106e5de928884d8bd78f5ce3e2dedd1ea99b225901b4a971f45c5f917b6c9"} Sep 15 12:36:01.485257 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:01.485232 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82" Sep 15 12:36:01.585722 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:01.585685 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71510952-7fbd-45c5-99bb-75aff4a4bb94-maas-api-service-ca\") pod \"71510952-7fbd-45c5-99bb-75aff4a4bb94\" (UID: \"71510952-7fbd-45c5-99bb-75aff4a4bb94\") " Sep 15 12:36:01.586060 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:01.586028 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71510952-7fbd-45c5-99bb-75aff4a4bb94-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71510952-7fbd-45c5-99bb-75aff4a4bb94" (UID: "71510952-7fbd-45c5-99bb-75aff4a4bb94"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:36:01.687609 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:01.687515 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71510952-7fbd-45c5-99bb-75aff4a4bb94-maas-api-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:02.356081 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:02.356046 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82" event={"ID":"71510952-7fbd-45c5-99bb-75aff4a4bb94","Type":"ContainerDied","Data":"a4a106e5de928884d8bd78f5ce3e2dedd1ea99b225901b4a971f45c5f917b6c9"} Sep 15 12:36:02.356081 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:02.356080 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82" Sep 15 12:36:02.356304 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:02.356098 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4a106e5de928884d8bd78f5ce3e2dedd1ea99b225901b4a971f45c5f917b6c9" Sep 15 12:36:09.562245 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.562209 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5"] Sep 15 12:36:09.562766 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.562454 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" podUID="181123e6-283f-481d-af02-0ca0c983638e" containerName="payload-processing" containerID="cri-o://e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208" gracePeriod=30 Sep 15 12:36:09.727989 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.727948 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82"] Sep 15 12:36:09.733459 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.733435 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7b204e-frv82"] Sep 15 12:36:09.815698 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.815629 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:36:09.854871 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.854835 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzd7c\" (UniqueName: \"kubernetes.io/projected/181123e6-283f-481d-af02-0ca0c983638e-kube-api-access-mzd7c\") pod \"181123e6-283f-481d-af02-0ca0c983638e\" (UID: \"181123e6-283f-481d-af02-0ca0c983638e\") " Sep 15 12:36:09.855031 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.854957 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/181123e6-283f-481d-af02-0ca0c983638e-plugins-config-volume\") pod \"181123e6-283f-481d-af02-0ca0c983638e\" (UID: \"181123e6-283f-481d-af02-0ca0c983638e\") " Sep 15 12:36:09.855421 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.855387 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/181123e6-283f-481d-af02-0ca0c983638e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "181123e6-283f-481d-af02-0ca0c983638e" (UID: "181123e6-283f-481d-af02-0ca0c983638e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:36:09.857176 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.857149 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/181123e6-283f-481d-af02-0ca0c983638e-kube-api-access-mzd7c" (OuterVolumeSpecName: "kube-api-access-mzd7c") pod "181123e6-283f-481d-af02-0ca0c983638e" (UID: "181123e6-283f-481d-af02-0ca0c983638e"). InnerVolumeSpecName "kube-api-access-mzd7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:36:09.956374 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.956334 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzd7c\" (UniqueName: \"kubernetes.io/projected/181123e6-283f-481d-af02-0ca0c983638e-kube-api-access-mzd7c\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:09.956374 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:09.956367 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/181123e6-283f-481d-af02-0ca0c983638e-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:10.398418 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.398388 2582 generic.go:356] "Generic (PLEG): container finished" podID="181123e6-283f-481d-af02-0ca0c983638e" containerID="e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208" exitCode=0 Sep 15 12:36:10.398637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.398434 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" Sep 15 12:36:10.398637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.398446 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" event={"ID":"181123e6-283f-481d-af02-0ca0c983638e","Type":"ContainerDied","Data":"e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208"} Sep 15 12:36:10.398637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.398529 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5" event={"ID":"181123e6-283f-481d-af02-0ca0c983638e","Type":"ContainerDied","Data":"3a0ecb7aad3e2dc807c018273a49c61b9bc51819e18627006c035f3fefb51d6e"} Sep 15 12:36:10.398637 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.398563 2582 scope.go:117] "RemoveContainer" containerID="e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208" Sep 15 12:36:10.408376 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.408360 2582 scope.go:117] "RemoveContainer" containerID="e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208" Sep 15 12:36:10.408774 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:36:10.408743 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208\": container with ID starting with e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208 not found: ID does not exist" containerID="e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208" Sep 15 12:36:10.408889 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.408777 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208"} err="failed to get container status \"e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208\": rpc error: code = NotFound desc = could not find container \"e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208\": container with ID starting with e38ad851b486db7fdb294b3ece9b485788ecc8d230fa855ecca39a586469c208 not found: ID does not exist" Sep 15 12:36:10.430625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.430590 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5"] Sep 15 12:36:10.437285 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.437244 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-7b204e-7848586f75-8ptr5"] Sep 15 12:36:10.711526 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.711405 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="181123e6-283f-481d-af02-0ca0c983638e" path="/var/lib/kubelet/pods/181123e6-283f-481d-af02-0ca0c983638e/volumes" Sep 15 12:36:10.711999 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:10.711913 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71510952-7fbd-45c5-99bb-75aff4a4bb94" path="/var/lib/kubelet/pods/71510952-7fbd-45c5-99bb-75aff4a4bb94/volumes" Sep 15 12:36:16.547320 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.547276 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s"] Sep 15 12:36:16.547926 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.547905 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" Sep 15 12:36:16.548066 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.548039 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbec5ae9-57ec-41cd-85e0-0b11bc240e1d" containerName="istio-proxy" Sep 15 12:36:16.548066 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.548070 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181123e6-283f-481d-af02-0ca0c983638e" containerName="payload-processing" Sep 15 12:36:16.548228 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.548080 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="181123e6-283f-481d-af02-0ca0c983638e" containerName="payload-processing" Sep 15 12:36:16.548228 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.548102 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71510952-7fbd-45c5-99bb-75aff4a4bb94" containerName="revoke-keys" Sep 15 12:36:16.548228 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.548111 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="71510952-7fbd-45c5-99bb-75aff4a4bb94" containerName="revoke-keys" Sep 15 12:36:16.548332 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.548241 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="181123e6-283f-481d-af02-0ca0c983638e" containerName="payload-processing" Sep 15 12:36:16.548332 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.548254 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="71510952-7fbd-45c5-99bb-75aff4a4bb94" containerName="revoke-keys" Sep 15 12:36:16.552729 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.552713 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" Sep 15 12:36:16.556136 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.556113 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:36:16.556325 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.556113 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:36:16.557614 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.557591 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s"] Sep 15 12:36:16.618431 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.618390 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d213a63-5eee-4a32-a34a-fa7ff2cd5498-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s\" (UID: \"9d213a63-5eee-4a32-a34a-fa7ff2cd5498\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" Sep 15 12:36:16.719975 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.719930 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d213a63-5eee-4a32-a34a-fa7ff2cd5498-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s\" (UID: \"9d213a63-5eee-4a32-a34a-fa7ff2cd5498\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" Sep 15 12:36:16.720570 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.720550 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d213a63-5eee-4a32-a34a-fa7ff2cd5498-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s\" (UID: \"9d213a63-5eee-4a32-a34a-fa7ff2cd5498\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" Sep 15 12:36:16.865109 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:16.865026 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" Sep 15 12:36:17.136891 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:17.136863 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s"] Sep 15 12:36:17.138762 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:36:17.138732 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d213a63_5eee_4a32_a34a_fa7ff2cd5498.slice/crio-35b00fc700b09fd800a01497f449ae798201db3a551740c6be313155c81ce52e WatchSource:0}: Error finding container 35b00fc700b09fd800a01497f449ae798201db3a551740c6be313155c81ce52e: Status 404 returned error can't find the container with id 35b00fc700b09fd800a01497f449ae798201db3a551740c6be313155c81ce52e Sep 15 12:36:17.437511 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:17.437389 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" event={"ID":"9d213a63-5eee-4a32-a34a-fa7ff2cd5498","Type":"ContainerStarted","Data":"5d1610194ba5bf8313a5b2a1b1ab7b4d8448961e0c86a15a10ba8a091ca8e22f"} Sep 15 12:36:17.437511 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:17.437455 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" event={"ID":"9d213a63-5eee-4a32-a34a-fa7ff2cd5498","Type":"ContainerStarted","Data":"35b00fc700b09fd800a01497f449ae798201db3a551740c6be313155c81ce52e"} Sep 15 12:36:17.452674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:17.452607 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" podStartSLOduration=1.452586698 podStartE2EDuration="1.452586698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:36:17.451568264 +0000 UTC m=+2029.208673336" watchObservedRunningTime="2026-09-15 12:36:17.452586698 +0000 UTC m=+2029.209691770" Sep 15 12:36:18.444695 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:18.444659 2582 generic.go:356] "Generic (PLEG): container finished" podID="9d213a63-5eee-4a32-a34a-fa7ff2cd5498" containerID="5d1610194ba5bf8313a5b2a1b1ab7b4d8448961e0c86a15a10ba8a091ca8e22f" exitCode=0 Sep 15 12:36:18.445091 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:18.444707 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" event={"ID":"9d213a63-5eee-4a32-a34a-fa7ff2cd5498","Type":"ContainerDied","Data":"5d1610194ba5bf8313a5b2a1b1ab7b4d8448961e0c86a15a10ba8a091ca8e22f"} Sep 15 12:36:19.586300 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:19.586273 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" Sep 15 12:36:19.649947 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:19.649910 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d213a63-5eee-4a32-a34a-fa7ff2cd5498-maas-api-service-ca\") pod \"9d213a63-5eee-4a32-a34a-fa7ff2cd5498\" (UID: \"9d213a63-5eee-4a32-a34a-fa7ff2cd5498\") " Sep 15 12:36:19.650275 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:19.650250 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d213a63-5eee-4a32-a34a-fa7ff2cd5498-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d213a63-5eee-4a32-a34a-fa7ff2cd5498" (UID: "9d213a63-5eee-4a32-a34a-fa7ff2cd5498"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:36:19.750967 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:19.750930 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d213a63-5eee-4a32-a34a-fa7ff2cd5498-maas-api-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:20.457994 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:20.457956 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" event={"ID":"9d213a63-5eee-4a32-a34a-fa7ff2cd5498","Type":"ContainerDied","Data":"35b00fc700b09fd800a01497f449ae798201db3a551740c6be313155c81ce52e"} Sep 15 12:36:20.457994 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:20.457982 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s" Sep 15 12:36:20.458215 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:20.458011 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35b00fc700b09fd800a01497f449ae798201db3a551740c6be313155c81ce52e" Sep 15 12:36:27.123539 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.123470 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq"] Sep 15 12:36:27.125424 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.125400 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1dc430-29824590-vc7zq"] Sep 15 12:36:27.157416 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.157383 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc"] Sep 15 12:36:27.157650 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.157625 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" podUID="1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11" containerName="payload-pre-processing" containerID="cri-o://da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e" gracePeriod=30 Sep 15 12:36:27.310332 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.310300 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s"] Sep 15 12:36:27.315820 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.315783 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1dc430-q775s"] Sep 15 12:36:27.423714 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.423684 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:36:27.496475 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.496441 2582 generic.go:356] "Generic (PLEG): container finished" podID="1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11" containerID="da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e" exitCode=0 Sep 15 12:36:27.496674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.496519 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" Sep 15 12:36:27.496674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.496523 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" event={"ID":"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11","Type":"ContainerDied","Data":"da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e"} Sep 15 12:36:27.496674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.496587 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc" event={"ID":"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11","Type":"ContainerDied","Data":"54e7106d95717a29f97af65fb996fe8d93f8dac96bd934d4538b39a6b2c19760"} Sep 15 12:36:27.496674 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.496620 2582 scope.go:117] "RemoveContainer" containerID="da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e" Sep 15 12:36:27.507102 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.507075 2582 scope.go:117] "RemoveContainer" containerID="da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e" Sep 15 12:36:27.507479 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:36:27.507449 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e\": container with ID starting with da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e not found: ID does not exist" containerID="da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e" Sep 15 12:36:27.507479 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.507514 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e"} err="failed to get container status \"da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e\": rpc error: code = NotFound desc = could not find container \"da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e\": container with ID starting with da42795dbf80076677e017cc78b40b7887ef4f1ef38979917f4d2b54c387580e not found: ID does not exist" Sep 15 12:36:27.526862 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.526830 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-plugins-config-volume\") pod \"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11\" (UID: \"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11\") " Sep 15 12:36:27.527022 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.526901 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9n52f\" (UniqueName: \"kubernetes.io/projected/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-kube-api-access-9n52f\") pod \"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11\" (UID: \"1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11\") " Sep 15 12:36:27.527309 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.527285 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11" (UID: "1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:36:27.529211 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.529189 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-kube-api-access-9n52f" (OuterVolumeSpecName: "kube-api-access-9n52f") pod "1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11" (UID: "1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11"). InnerVolumeSpecName "kube-api-access-9n52f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:36:27.628077 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.628035 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:27.628077 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.628067 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n52f\" (UniqueName: \"kubernetes.io/projected/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11-kube-api-access-9n52f\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:27.825291 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.825255 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc"] Sep 15 12:36:27.829457 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:27.829422 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1dc430-86c7cd6678-dfmzc"] Sep 15 12:36:28.711611 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:28.711580 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11" path="/var/lib/kubelet/pods/1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11/volumes" Sep 15 12:36:28.711973 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:28.711884 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77a7581f-3035-4803-881d-a884a9a1111d" path="/var/lib/kubelet/pods/77a7581f-3035-4803-881d-a884a9a1111d/volumes" Sep 15 12:36:28.712127 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:28.712115 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d213a63-5eee-4a32-a34a-fa7ff2cd5498" path="/var/lib/kubelet/pods/9d213a63-5eee-4a32-a34a-fa7ff2cd5498/volumes" Sep 15 12:36:29.078895 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:29.078864 2582 scope.go:117] "RemoveContainer" containerID="07a19a8c780f531b3346553dd1009b2b09280fa4e5a7bb0d0360831059d83fc0" Sep 15 12:36:34.271833 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.271789 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf"] Sep 15 12:36:34.272323 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.272203 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d213a63-5eee-4a32-a34a-fa7ff2cd5498" containerName="revoke-keys" Sep 15 12:36:34.272323 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.272215 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d213a63-5eee-4a32-a34a-fa7ff2cd5498" containerName="revoke-keys" Sep 15 12:36:34.272323 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.272227 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11" containerName="payload-pre-processing" Sep 15 12:36:34.272323 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.272233 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11" containerName="payload-pre-processing" Sep 15 12:36:34.272323 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.272291 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d213a63-5eee-4a32-a34a-fa7ff2cd5498" containerName="revoke-keys" Sep 15 12:36:34.272323 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.272300 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ddee2bb-0d48-41c7-ac7d-3bf8420f4f11" containerName="payload-pre-processing" Sep 15 12:36:34.275312 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.275292 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" Sep 15 12:36:34.277957 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.277933 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tbvkt\"" Sep 15 12:36:34.278093 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.277933 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:36:34.285184 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.285157 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf"] Sep 15 12:36:34.398132 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.398092 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf\" (UID: \"49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" Sep 15 12:36:34.498988 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.498945 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf\" (UID: \"49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" Sep 15 12:36:34.499619 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.499600 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf\" (UID: \"49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" Sep 15 12:36:34.587096 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.587006 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" Sep 15 12:36:34.864198 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:34.864172 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf"] Sep 15 12:36:34.866218 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:36:34.866180 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f953ec_4ff7_4c2a_a79d_1e84fdfbbc4d.slice/crio-22a23fed106f9fc7ee00a590dbac950f20f35e4d6a04c5e0dfffa42cdd290c8a WatchSource:0}: Error finding container 22a23fed106f9fc7ee00a590dbac950f20f35e4d6a04c5e0dfffa42cdd290c8a: Status 404 returned error can't find the container with id 22a23fed106f9fc7ee00a590dbac950f20f35e4d6a04c5e0dfffa42cdd290c8a Sep 15 12:36:35.535531 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:35.535466 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" event={"ID":"49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d","Type":"ContainerStarted","Data":"a99f90a64762ccdc3d5116c6a5a6dcbbe72ab57b652a638de6c2430cec5f6c32"} Sep 15 12:36:35.535930 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:35.535556 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" event={"ID":"49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d","Type":"ContainerStarted","Data":"22a23fed106f9fc7ee00a590dbac950f20f35e4d6a04c5e0dfffa42cdd290c8a"} Sep 15 12:36:35.553111 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:35.553054 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" podStartSLOduration=1.553037883 podStartE2EDuration="1.553037883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:36:35.550963881 +0000 UTC m=+2047.308068953" watchObservedRunningTime="2026-09-15 12:36:35.553037883 +0000 UTC m=+2047.310142991" Sep 15 12:36:36.541949 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:36.541914 2582 generic.go:356] "Generic (PLEG): container finished" podID="49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d" containerID="a99f90a64762ccdc3d5116c6a5a6dcbbe72ab57b652a638de6c2430cec5f6c32" exitCode=0 Sep 15 12:36:36.542336 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:36.541999 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" event={"ID":"49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d","Type":"ContainerDied","Data":"a99f90a64762ccdc3d5116c6a5a6dcbbe72ab57b652a638de6c2430cec5f6c32"} Sep 15 12:36:37.684252 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:37.684221 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" Sep 15 12:36:37.832859 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:37.832762 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d-maas-api-service-ca\") pod \"49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d\" (UID: \"49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d\") " Sep 15 12:36:37.833142 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:37.833119 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d" (UID: "49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:36:37.933945 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:37.933901 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d-maas-api-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:38.553824 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:38.553781 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" event={"ID":"49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d","Type":"ContainerDied","Data":"22a23fed106f9fc7ee00a590dbac950f20f35e4d6a04c5e0dfffa42cdd290c8a"} Sep 15 12:36:38.553824 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:38.553809 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf" Sep 15 12:36:38.554064 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:38.553881 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22a23fed106f9fc7ee00a590dbac950f20f35e4d6a04c5e0dfffa42cdd290c8a" Sep 15 12:36:44.780420 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:44.780382 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx"] Sep 15 12:36:44.780957 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:44.780723 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" podUID="d4858017-7123-467f-98dc-ecfa71491749" containerName="payload-processing" containerID="cri-o://341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327" gracePeriod=30 Sep 15 12:36:44.790476 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:44.790346 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p"] Sep 15 12:36:44.790705 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:44.790633 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" podUID="e2468531-ddb4-40ca-9d3a-9c884fe24d8a" containerName="payload-pre-processing" containerID="cri-o://b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d" gracePeriod=30 Sep 15 12:36:44.944431 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:44.944391 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf"] Sep 15 12:36:44.947397 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:44.947344 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-969130-ss6lf"] Sep 15 12:36:45.065551 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.065528 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:36:45.068523 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.068469 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:36:45.107676 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.106953 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4858017-7123-467f-98dc-ecfa71491749-plugins-config-volume\") pod \"d4858017-7123-467f-98dc-ecfa71491749\" (UID: \"d4858017-7123-467f-98dc-ecfa71491749\") " Sep 15 12:36:45.107676 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.107064 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkkl7\" (UniqueName: \"kubernetes.io/projected/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-kube-api-access-xkkl7\") pod \"e2468531-ddb4-40ca-9d3a-9c884fe24d8a\" (UID: \"e2468531-ddb4-40ca-9d3a-9c884fe24d8a\") " Sep 15 12:36:45.107676 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.107139 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-plugins-config-volume\") pod \"e2468531-ddb4-40ca-9d3a-9c884fe24d8a\" (UID: \"e2468531-ddb4-40ca-9d3a-9c884fe24d8a\") " Sep 15 12:36:45.107676 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.107169 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khsxh\" (UniqueName: \"kubernetes.io/projected/d4858017-7123-467f-98dc-ecfa71491749-kube-api-access-khsxh\") pod \"d4858017-7123-467f-98dc-ecfa71491749\" (UID: \"d4858017-7123-467f-98dc-ecfa71491749\") " Sep 15 12:36:45.108707 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.108476 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4858017-7123-467f-98dc-ecfa71491749-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4858017-7123-467f-98dc-ecfa71491749" (UID: "d4858017-7123-467f-98dc-ecfa71491749"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:36:45.109079 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.109048 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2468531-ddb4-40ca-9d3a-9c884fe24d8a" (UID: "e2468531-ddb4-40ca-9d3a-9c884fe24d8a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:36:45.110378 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.110351 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-kube-api-access-xkkl7" (OuterVolumeSpecName: "kube-api-access-xkkl7") pod "e2468531-ddb4-40ca-9d3a-9c884fe24d8a" (UID: "e2468531-ddb4-40ca-9d3a-9c884fe24d8a"). InnerVolumeSpecName "kube-api-access-xkkl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:36:45.110595 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.110567 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4858017-7123-467f-98dc-ecfa71491749-kube-api-access-khsxh" (OuterVolumeSpecName: "kube-api-access-khsxh") pod "d4858017-7123-467f-98dc-ecfa71491749" (UID: "d4858017-7123-467f-98dc-ecfa71491749"). InnerVolumeSpecName "kube-api-access-khsxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:36:45.208285 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.208245 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4858017-7123-467f-98dc-ecfa71491749-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:45.208285 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.208275 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkkl7\" (UniqueName: \"kubernetes.io/projected/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-kube-api-access-xkkl7\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:45.208285 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.208284 2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2468531-ddb4-40ca-9d3a-9c884fe24d8a-plugins-config-volume\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:45.208285 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.208293 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khsxh\" (UniqueName: \"kubernetes.io/projected/d4858017-7123-467f-98dc-ecfa71491749-kube-api-access-khsxh\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:36:45.591546 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.591472 2582 generic.go:356] "Generic (PLEG): container finished" podID="d4858017-7123-467f-98dc-ecfa71491749" containerID="341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327" exitCode=0 Sep 15 12:36:45.591763 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.591693 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" event={"ID":"d4858017-7123-467f-98dc-ecfa71491749","Type":"ContainerDied","Data":"341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327"} Sep 15 12:36:45.591836 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.591785 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" event={"ID":"d4858017-7123-467f-98dc-ecfa71491749","Type":"ContainerDied","Data":"c1e3adc287589b0a0aaba326c66d1c6d2fab0bbd4281965efc372a1906eba651"} Sep 15 12:36:45.591836 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.591822 2582 scope.go:117] "RemoveContainer" containerID="341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327" Sep 15 12:36:45.592407 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.592007 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx" Sep 15 12:36:45.598053 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.598022 2582 generic.go:356] "Generic (PLEG): container finished" podID="e2468531-ddb4-40ca-9d3a-9c884fe24d8a" containerID="b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d" exitCode=0 Sep 15 12:36:45.598165 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.598057 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" event={"ID":"e2468531-ddb4-40ca-9d3a-9c884fe24d8a","Type":"ContainerDied","Data":"b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d"} Sep 15 12:36:45.598165 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.598088 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" Sep 15 12:36:45.598165 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.598124 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p" event={"ID":"e2468531-ddb4-40ca-9d3a-9c884fe24d8a","Type":"ContainerDied","Data":"023d11163acb97bae2131d5b74e464e09574d56a00632af245a578e4c28567b2"} Sep 15 12:36:45.604431 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.604410 2582 scope.go:117] "RemoveContainer" containerID="341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327" Sep 15 12:36:45.604859 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:36:45.604842 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327\": container with ID starting with 341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327 not found: ID does not exist" containerID="341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327" Sep 15 12:36:45.604927 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.604867 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327"} err="failed to get container status \"341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327\": rpc error: code = NotFound desc = could not find container \"341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327\": container with ID starting with 341359f5b6ff006dd655c162d72e4e3439ebfab00e87c013302ed41a9da96327 not found: ID does not exist" Sep 15 12:36:45.604927 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.604883 2582 scope.go:117] "RemoveContainer" containerID="b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d" Sep 15 12:36:45.613364 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.613343 2582 scope.go:117] "RemoveContainer" containerID="b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d" Sep 15 12:36:45.613809 ip-10-0-130-139 kubenswrapper[2582]: E0915 12:36:45.613752 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d\": container with ID starting with b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d not found: ID does not exist" containerID="b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d" Sep 15 12:36:45.613880 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.613816 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d"} err="failed to get container status \"b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d\": rpc error: code = NotFound desc = could not find container \"b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d\": container with ID starting with b660712f639f21c3c1f3b9e4ae7f8a3b110739c5133bd144f37d571d8744c18d not found: ID does not exist" Sep 15 12:36:45.616590 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.616568 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx"] Sep 15 12:36:45.620759 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.620739 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-969130-54477f6c65-dn4sx"] Sep 15 12:36:45.631504 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.631461 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p"] Sep 15 12:36:45.638395 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:45.638360 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-969130-69d7578698-dq22p"] Sep 15 12:36:46.711974 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:46.711938 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d" path="/var/lib/kubelet/pods/49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d/volumes" Sep 15 12:36:46.712354 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:46.712287 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4858017-7123-467f-98dc-ecfa71491749" path="/var/lib/kubelet/pods/d4858017-7123-467f-98dc-ecfa71491749/volumes" Sep 15 12:36:46.712617 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:36:46.712603 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2468531-ddb4-40ca-9d3a-9c884fe24d8a" path="/var/lib/kubelet/pods/e2468531-ddb4-40ca-9d3a-9c884fe24d8a/volumes" Sep 15 12:41:29.147261 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:41:29.147231 2582 scope.go:117] "RemoveContainer" containerID="5b423d11f3d22430b8099938a521a10073d642a41d1ddf9403f38b3314550220" Sep 15 12:42:25.454123 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.454089 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-865999984f-rjcws"] Sep 15 12:42:25.454592 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.454451 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2468531-ddb4-40ca-9d3a-9c884fe24d8a" containerName="payload-pre-processing" Sep 15 12:42:25.454592 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.454462 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2468531-ddb4-40ca-9d3a-9c884fe24d8a" containerName="payload-pre-processing" Sep 15 12:42:25.454592 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.454497 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4858017-7123-467f-98dc-ecfa71491749" containerName="payload-processing" Sep 15 12:42:25.454592 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.454504 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4858017-7123-467f-98dc-ecfa71491749" containerName="payload-processing" Sep 15 12:42:25.454592 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.454513 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d" containerName="revoke-keys" Sep 15 12:42:25.454592 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.454519 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d" containerName="revoke-keys" Sep 15 12:42:25.454592 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.454587 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4858017-7123-467f-98dc-ecfa71491749" containerName="payload-processing" Sep 15 12:42:25.454592 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.454595 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="49f953ec-4ff7-4c2a-a79d-1e84fdfbbc4d" containerName="revoke-keys" Sep 15 12:42:25.454857 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.454603 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2468531-ddb4-40ca-9d3a-9c884fe24d8a" containerName="payload-pre-processing" Sep 15 12:42:25.457853 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.457836 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.460615 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.460581 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 15 12:42:25.461731 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.461703 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 15 12:42:25.461867 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.461750 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 15 12:42:25.461867 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.461770 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 12:42:25.461867 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.461753 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jhxcj\"" Sep 15 12:42:25.470930 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.470903 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-865999984f-rjcws"] Sep 15 12:42:25.575410 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.575370 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-openshift-service-ca\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.575638 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.575478 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c78z\" (UniqueName: \"kubernetes.io/projected/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-kube-api-access-7c78z\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.575638 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.575530 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-metrics-certs\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.575638 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.575550 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-maas-api-tls\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.676544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.676472 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c78z\" (UniqueName: \"kubernetes.io/projected/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-kube-api-access-7c78z\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.676736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.676573 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-metrics-certs\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.676736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.676609 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-maas-api-tls\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.676736 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.676689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-openshift-service-ca\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.677304 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.677284 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-openshift-service-ca\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.679269 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.679246 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-maas-api-tls\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.679391 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.679281 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-metrics-certs\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.683979 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.683954 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c78z\" (UniqueName: \"kubernetes.io/projected/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-kube-api-access-7c78z\") pod \"maas-api-865999984f-rjcws\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:25.768320 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:25.768262 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:26.042841 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:26.042818 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-865999984f-rjcws"] Sep 15 12:42:26.045243 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:42:26.045218 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb7892d_5881_4a3f_8ed5_9926e8f523a5.slice/crio-8f34e330cf44d9b5fb2ee22d3fc7c84334082406f4f0de6156fa15159aec7721 WatchSource:0}: Error finding container 8f34e330cf44d9b5fb2ee22d3fc7c84334082406f4f0de6156fa15159aec7721: Status 404 returned error can't find the container with id 8f34e330cf44d9b5fb2ee22d3fc7c84334082406f4f0de6156fa15159aec7721 Sep 15 12:42:26.046958 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:26.046938 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 12:42:26.319644 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:26.319557 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" event={"ID":"eeb7892d-5881-4a3f-8ed5-9926e8f523a5","Type":"ContainerStarted","Data":"8f34e330cf44d9b5fb2ee22d3fc7c84334082406f4f0de6156fa15159aec7721"} Sep 15 12:42:29.167012 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:29.166983 2582 scope.go:117] "RemoveContainer" containerID="ce68a92fd4c1d1f467d66d57db8c648e7e369d2dbc0b3bb3910a7fe0513b44d0" Sep 15 12:42:29.381111 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:29.381078 2582 scope.go:117] "RemoveContainer" containerID="5d1610194ba5bf8313a5b2a1b1ab7b4d8448961e0c86a15a10ba8a091ca8e22f" Sep 15 12:42:30.342198 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:30.342159 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" event={"ID":"eeb7892d-5881-4a3f-8ed5-9926e8f523a5","Type":"ContainerStarted","Data":"d7cab6780e720e7e8d15a4a784443cdd3b5262c05c03d057365cf674ba524d0e"} Sep 15 12:42:30.342647 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:30.342298 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:30.360783 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:30.360731 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" podStartSLOduration=1.97075602 podStartE2EDuration="5.36071563s" totalImagesPullingTime="3.38995961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:42:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:42:30.36037373 +0000 UTC m=+2402.117478801" watchObservedRunningTime="2026-09-15 12:42:30.36071563 +0000 UTC m=+2402.117820733" Sep 15 12:42:33.151819 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.151781 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-865999984f-rjcws"] Sep 15 12:42:33.152276 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.152013 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" podUID="eeb7892d-5881-4a3f-8ed5-9926e8f523a5" containerName="maas-api" containerID="cri-o://d7cab6780e720e7e8d15a4a784443cdd3b5262c05c03d057365cf674ba524d0e" gracePeriod=30 Sep 15 12:42:33.181922 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.181894 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp"] Sep 15 12:42:33.185250 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.185233 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.194698 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.194665 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp"] Sep 15 12:42:33.253790 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.253757 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dc4cad-5380-4200-b88e-b018eb092eb8-openshift-service-ca\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.253973 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.253818 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1dc4cad-5380-4200-b88e-b018eb092eb8-maas-api-tls\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.253973 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.253850 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbw9c\" (UniqueName: \"kubernetes.io/projected/e1dc4cad-5380-4200-b88e-b018eb092eb8-kube-api-access-bbw9c\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.253973 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.253891 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1dc4cad-5380-4200-b88e-b018eb092eb8-metrics-certs\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.355110 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.355077 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dc4cad-5380-4200-b88e-b018eb092eb8-openshift-service-ca\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.355303 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.355123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1dc4cad-5380-4200-b88e-b018eb092eb8-maas-api-tls\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.355303 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.355237 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbw9c\" (UniqueName: \"kubernetes.io/projected/e1dc4cad-5380-4200-b88e-b018eb092eb8-kube-api-access-bbw9c\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.355303 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.355282 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1dc4cad-5380-4200-b88e-b018eb092eb8-metrics-certs\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.355860 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.355831 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dc4cad-5380-4200-b88e-b018eb092eb8-openshift-service-ca\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.358027 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.357998 2582 generic.go:356] "Generic (PLEG): container finished" podID="eeb7892d-5881-4a3f-8ed5-9926e8f523a5" containerID="d7cab6780e720e7e8d15a4a784443cdd3b5262c05c03d057365cf674ba524d0e" exitCode=0 Sep 15 12:42:33.358027 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.358023 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" event={"ID":"eeb7892d-5881-4a3f-8ed5-9926e8f523a5","Type":"ContainerDied","Data":"d7cab6780e720e7e8d15a4a784443cdd3b5262c05c03d057365cf674ba524d0e"} Sep 15 12:42:33.358215 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.358086 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1dc4cad-5380-4200-b88e-b018eb092eb8-maas-api-tls\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.358276 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.358233 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1dc4cad-5380-4200-b88e-b018eb092eb8-metrics-certs\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.364283 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.364259 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbw9c\" (UniqueName: \"kubernetes.io/projected/e1dc4cad-5380-4200-b88e-b018eb092eb8-kube-api-access-bbw9c\") pod \"maas-api-5dd5d756d5-jjkwp\" (UID: \"e1dc4cad-5380-4200-b88e-b018eb092eb8\") " pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.395073 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.395045 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:33.534578 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.534539 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:33.557577 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.557545 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7c78z\" (UniqueName: \"kubernetes.io/projected/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-kube-api-access-7c78z\") pod \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " Sep 15 12:42:33.557577 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.557592 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-openshift-service-ca\") pod \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " Sep 15 12:42:33.557922 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.557618 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-maas-api-tls\") pod \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " Sep 15 12:42:33.557990 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.557947 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eeb7892d-5881-4a3f-8ed5-9926e8f523a5" (UID: "eeb7892d-5881-4a3f-8ed5-9926e8f523a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 12:42:33.558060 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.558003 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-metrics-certs\") pod \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\" (UID: \"eeb7892d-5881-4a3f-8ed5-9926e8f523a5\") " Sep 15 12:42:33.558427 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.558381 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-openshift-service-ca\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:42:33.560743 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.560707 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eeb7892d-5881-4a3f-8ed5-9926e8f523a5" (UID: "eeb7892d-5881-4a3f-8ed5-9926e8f523a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:42:33.560869 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.560770 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eeb7892d-5881-4a3f-8ed5-9926e8f523a5" (UID: "eeb7892d-5881-4a3f-8ed5-9926e8f523a5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 12:42:33.560947 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.560930 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-kube-api-access-7c78z" (OuterVolumeSpecName: "kube-api-access-7c78z") pod "eeb7892d-5881-4a3f-8ed5-9926e8f523a5" (UID: "eeb7892d-5881-4a3f-8ed5-9926e8f523a5"). InnerVolumeSpecName "kube-api-access-7c78z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 12:42:33.659133 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.659102 2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-metrics-certs\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:42:33.659133 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.659131 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7c78z\" (UniqueName: \"kubernetes.io/projected/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-kube-api-access-7c78z\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:42:33.659133 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.659141 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeb7892d-5881-4a3f-8ed5-9926e8f523a5-maas-api-tls\") on node \"ip-10-0-130-139.ec2.internal\" DevicePath \"\"" Sep 15 12:42:33.813310 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:33.813284 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp"] Sep 15 12:42:33.815555 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:42:33.815519 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1dc4cad_5380_4200_b88e_b018eb092eb8.slice/crio-2025ab01895befbe9fb0e7327b8e504ef652db234e0820d783a32c30b3d63ad2 WatchSource:0}: Error finding container 2025ab01895befbe9fb0e7327b8e504ef652db234e0820d783a32c30b3d63ad2: Status 404 returned error can't find the container with id 2025ab01895befbe9fb0e7327b8e504ef652db234e0820d783a32c30b3d63ad2 Sep 15 12:42:34.366290 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:34.366245 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" event={"ID":"eeb7892d-5881-4a3f-8ed5-9926e8f523a5","Type":"ContainerDied","Data":"8f34e330cf44d9b5fb2ee22d3fc7c84334082406f4f0de6156fa15159aec7721"} Sep 15 12:42:34.366789 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:34.366336 2582 scope.go:117] "RemoveContainer" containerID="d7cab6780e720e7e8d15a4a784443cdd3b5262c05c03d057365cf674ba524d0e" Sep 15 12:42:34.366789 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:34.366278 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-865999984f-rjcws" Sep 15 12:42:34.367936 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:34.367771 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" event={"ID":"e1dc4cad-5380-4200-b88e-b018eb092eb8","Type":"ContainerStarted","Data":"2025ab01895befbe9fb0e7327b8e504ef652db234e0820d783a32c30b3d63ad2"} Sep 15 12:42:34.389057 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:34.389025 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-865999984f-rjcws"] Sep 15 12:42:34.392253 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:34.392227 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-865999984f-rjcws"] Sep 15 12:42:34.711621 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:34.711587 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeb7892d-5881-4a3f-8ed5-9926e8f523a5" path="/var/lib/kubelet/pods/eeb7892d-5881-4a3f-8ed5-9926e8f523a5/volumes" Sep 15 12:42:35.374713 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:35.374672 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" event={"ID":"e1dc4cad-5380-4200-b88e-b018eb092eb8","Type":"ContainerStarted","Data":"0fe66228d8abffbe5415bb48cd0d7ccc4015c9ad4444e5dc9eb96582fbd4cb4e"} Sep 15 12:42:35.375244 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:35.375220 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:42:35.398544 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:35.398477 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" podStartSLOduration=1.339840242 podStartE2EDuration="2.39846193s" totalImagesPullingTime="1.058621688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:42:35.396894811 +0000 UTC m=+2407.153999906" watchObservedRunningTime="2026-09-15 12:42:35.39846193 +0000 UTC m=+2407.155567004" Sep 15 12:42:41.394586 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:42:41.394555 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5dd5d756d5-jjkwp" Sep 15 12:43:05.371649 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:05.371619 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jqns7_26665ca0-d0ee-48aa-958f-add728583758/server/0.log" Sep 15 12:43:05.489293 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:05.489262 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-422xq_87db7543-c2aa-4a47-9363-9fdd047dafa4/manager/0.log" Sep 15 12:43:05.602176 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:05.602148 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b747b9585-74lgg_4dc67243-5de9-4de1-bc50-df52da5facf8/copy-manifests/0.log" Sep 15 12:43:05.620941 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:05.620907 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b747b9585-74lgg_4dc67243-5de9-4de1-bc50-df52da5facf8/manager/0.log" Sep 15 12:43:06.300135 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:06.300104 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5dd5d756d5-jjkwp_e1dc4cad-5380-4200-b88e-b018eb092eb8/maas-api/0.log" Sep 15 12:43:06.404615 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:06.404586 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29824575-lfcvf_6df38f6a-67bb-4d6c-b088-de84728e0db6/cleanup/0.log" Sep 15 12:43:06.514954 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:06.514926 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29824590-v89wm_62512f37-af98-4011-9d99-fbe7306ec1ce/cleanup/0.log" Sep 15 12:43:08.418850 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:08.418820 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv_5906647c-59ec-4391-8389-8dcc624b1b1f/istio-proxy/0.log" Sep 15 12:43:08.543236 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:08.543204 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ccvgl_091727bd-6619-4ee6-8be2-718eec0832ec/discovery/0.log" Sep 15 12:43:08.877080 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:08.877050 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-nv8bv_c61cf7af-1e83-4674-b1b5-8795276fd46f/istio-proxy/1.log" Sep 15 12:43:09.112619 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:09.112587 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-clfp7_bd363dc1-4a24-4aed-83d0-d5bcbdedd04f/payload-processing/0.log" Sep 15 12:43:09.571829 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:09.571794 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-sk25w_b3174195-cb86-409d-ba83-4f24db34a882/main/0.log" Sep 15 12:43:09.904597 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:09.904516 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-s8m2q_c62e7e15-1524-490f-b34b-505dfe2e45cf/main/0.log" Sep 15 12:43:10.018153 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:10.018121 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4vwf2_e3a19858-3f4e-4990-93e2-b0d62bbdf757/main/0.log" Sep 15 12:43:18.631629 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:18.631599 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dktw7_859d410c-5eea-4f7f-a913-f86efa9a74dc/global-pull-secret-syncer/0.log" Sep 15 12:43:18.796475 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:18.796446 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mm7pz_40abd97c-1d7a-4ff0-a8e1-23c8fb4805a7/konnectivity-agent/0.log" Sep 15 12:43:18.823267 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:18.823237 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-139.ec2.internal_187d0f2e9822248b4723c1d352f18f05/haproxy/0.log" Sep 15 12:43:25.641696 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:25.641627 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a3e584aa-8fdd-4a51-9324-67b86b6d615d/alertmanager/0.log" Sep 15 12:43:25.661285 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:25.661239 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a3e584aa-8fdd-4a51-9324-67b86b6d615d/config-reloader/0.log" Sep 15 12:43:25.679259 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:25.679232 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a3e584aa-8fdd-4a51-9324-67b86b6d615d/kube-rbac-proxy-web/0.log" Sep 15 12:43:25.697609 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:25.697575 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a3e584aa-8fdd-4a51-9324-67b86b6d615d/kube-rbac-proxy/0.log" Sep 15 12:43:25.716875 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:25.716842 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a3e584aa-8fdd-4a51-9324-67b86b6d615d/kube-rbac-proxy-metric/0.log" Sep 15 12:43:25.737335 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:25.737309 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a3e584aa-8fdd-4a51-9324-67b86b6d615d/prom-label-proxy/0.log" Sep 15 12:43:25.756134 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:25.756109 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a3e584aa-8fdd-4a51-9324-67b86b6d615d/init-config-reloader/0.log" Sep 15 12:43:25.890603 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:25.890520 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6ccf494cc7-zjzqk_141bbde0-f532-43df-b468-f006caafcc52/metrics-server/0.log" Sep 15 12:43:26.077027 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.076992 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lfvs8_7ef1b202-71f6-4738-9054-57aa0dea17fd/node-exporter/0.log" Sep 15 12:43:26.109683 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.109647 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lfvs8_7ef1b202-71f6-4738-9054-57aa0dea17fd/kube-rbac-proxy/0.log" Sep 15 12:43:26.133620 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.133592 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lfvs8_7ef1b202-71f6-4738-9054-57aa0dea17fd/init-textfile/0.log" Sep 15 12:43:26.258426 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.258379 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-47rvs_ef3d2d84-b9a3-4983-b3da-a7732cf7c12a/kube-rbac-proxy-main/0.log" Sep 15 12:43:26.280906 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.280879 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-47rvs_ef3d2d84-b9a3-4983-b3da-a7732cf7c12a/kube-rbac-proxy-self/0.log" Sep 15 12:43:26.301595 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.301560 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-47rvs_ef3d2d84-b9a3-4983-b3da-a7732cf7c12a/openshift-state-metrics/0.log" Sep 15 12:43:26.346383 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.346351 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7348ef62-d03a-4a28-9411-9f975a6bab60/prometheus/0.log" Sep 15 12:43:26.365419 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.365383 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7348ef62-d03a-4a28-9411-9f975a6bab60/config-reloader/0.log" Sep 15 12:43:26.384099 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.384070 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7348ef62-d03a-4a28-9411-9f975a6bab60/thanos-sidecar/0.log" Sep 15 12:43:26.404092 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.404019 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7348ef62-d03a-4a28-9411-9f975a6bab60/kube-rbac-proxy-web/0.log" Sep 15 12:43:26.429535 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.429509 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7348ef62-d03a-4a28-9411-9f975a6bab60/kube-rbac-proxy/0.log" Sep 15 12:43:26.449916 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.449891 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7348ef62-d03a-4a28-9411-9f975a6bab60/kube-rbac-proxy-thanos/0.log" Sep 15 12:43:26.474796 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.474769 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7348ef62-d03a-4a28-9411-9f975a6bab60/init-config-reloader/0.log" Sep 15 12:43:26.509914 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.509885 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-hqwfr_96a6e508-7764-49da-8691-c28848eb19d4/prometheus-operator/0.log" Sep 15 12:43:26.532428 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:26.532403 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-hqwfr_96a6e508-7764-49da-8691-c28848eb19d4/kube-rbac-proxy/0.log" Sep 15 12:43:29.012097 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.012066 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m"] Sep 15 12:43:29.012694 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.012664 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeb7892d-5881-4a3f-8ed5-9926e8f523a5" containerName="maas-api" Sep 15 12:43:29.012694 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.012690 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeb7892d-5881-4a3f-8ed5-9926e8f523a5" containerName="maas-api" Sep 15 12:43:29.012917 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.012811 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeb7892d-5881-4a3f-8ed5-9926e8f523a5" containerName="maas-api" Sep 15 12:43:29.016352 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.016334 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.018893 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.018876 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n4f4n\"/\"kube-root-ca.crt\"" Sep 15 12:43:29.019888 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.019870 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n4f4n\"/\"openshift-service-ca.crt\"" Sep 15 12:43:29.019990 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.019872 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n4f4n\"/\"default-dockercfg-27rxc\"" Sep 15 12:43:29.026805 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.026785 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m"] Sep 15 12:43:29.103708 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.103672 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4sfb\" (UniqueName: \"kubernetes.io/projected/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-kube-api-access-g4sfb\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.103912 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.103722 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-sys\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.103912 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.103770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-proc\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.103912 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.103795 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-podres\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.104077 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.103923 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-lib-modules\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.204949 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.204916 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4sfb\" (UniqueName: \"kubernetes.io/projected/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-kube-api-access-g4sfb\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.204949 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.204953 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-sys\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.205161 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.204995 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-proc\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.205161 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.205013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-podres\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.205161 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.205105 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-lib-modules\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.205161 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.205105 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-proc\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.205297 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.205206 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-podres\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.205297 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.205223 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-lib-modules\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.205297 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.205209 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-sys\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.213944 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.213914 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4sfb\" (UniqueName: \"kubernetes.io/projected/cb0efcbc-889c-4ae8-a5b7-e990e37353a7-kube-api-access-g4sfb\") pod \"perf-node-gather-daemonset-m4h8m\" (UID: \"cb0efcbc-889c-4ae8-a5b7-e990e37353a7\") " pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.259948 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.259915 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7498d49ccd-7656n_a2694ca0-9aeb-4e74-9f82-abd1d9f4bb0e/console/0.log" Sep 15 12:43:29.327668 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.327573 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:29.435320 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.435286 2582 scope.go:117] "RemoveContainer" containerID="a99f90a64762ccdc3d5116c6a5a6dcbbe72ab57b652a638de6c2430cec5f6c32" Sep 15 12:43:29.601661 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.601636 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m"] Sep 15 12:43:29.603795 ip-10-0-130-139 kubenswrapper[2582]: W0915 12:43:29.603751 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcb0efcbc_889c_4ae8_a5b7_e990e37353a7.slice/crio-0f087b7745f358944fd98416f0f5c3f62a91af8fce6465965efece1c0b43e6ce WatchSource:0}: Error finding container 0f087b7745f358944fd98416f0f5c3f62a91af8fce6465965efece1c0b43e6ce: Status 404 returned error can't find the container with id 0f087b7745f358944fd98416f0f5c3f62a91af8fce6465965efece1c0b43e6ce Sep 15 12:43:29.654225 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:29.654196 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" event={"ID":"cb0efcbc-889c-4ae8-a5b7-e990e37353a7","Type":"ContainerStarted","Data":"0f087b7745f358944fd98416f0f5c3f62a91af8fce6465965efece1c0b43e6ce"} Sep 15 12:43:30.660794 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:30.660749 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" event={"ID":"cb0efcbc-889c-4ae8-a5b7-e990e37353a7","Type":"ContainerStarted","Data":"61b153d424705e04c6e55e71d831c5fc3513de11e8a9a2c94bf2a03f02352bb8"} Sep 15 12:43:30.661261 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:30.660877 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:30.678091 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:30.678038 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" podStartSLOduration=2.678017213 podStartE2EDuration="2.678017213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:43:30.676390034 +0000 UTC m=+2462.433495106" watchObservedRunningTime="2026-09-15 12:43:30.678017213 +0000 UTC m=+2462.435122283" Sep 15 12:43:31.219625 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:31.219592 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x5jkb_1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb/dns/0.log" Sep 15 12:43:31.242946 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:31.242918 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x5jkb_1acc9419-93ce-4bdb-8ac2-58ecc9d91bcb/kube-rbac-proxy/0.log" Sep 15 12:43:31.264207 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:31.264182 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5n22m_5b7beb0e-ce51-4c27-97b4-33be9eb3cf5b/dns-node-resolver/0.log" Sep 15 12:43:31.854935 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:31.854892 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pbjq_c2843e32-3281-4cc4-a0cb-1700a9929e5a/node-ca/0.log" Sep 15 12:43:32.807823 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:32.807789 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv_5906647c-59ec-4391-8389-8dcc624b1b1f/istio-proxy/0.log" Sep 15 12:43:33.632462 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:33.632425 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ccvgl_091727bd-6619-4ee6-8be2-718eec0832ec/discovery/0.log" Sep 15 12:43:33.705659 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:33.705629 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-nv8bv_c61cf7af-1e83-4674-b1b5-8795276fd46f/istio-proxy/0.log" Sep 15 12:43:33.710986 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:33.710961 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-nv8bv_c61cf7af-1e83-4674-b1b5-8795276fd46f/istio-proxy/1.log" Sep 15 12:43:33.791043 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:33.791011 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-clfp7_bd363dc1-4a24-4aed-83d0-d5bcbdedd04f/payload-processing/0.log" Sep 15 12:43:34.684286 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:34.684248 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rz5vx_14a320c1-cabe-447c-af4c-84fc10143b41/serve-healthcheck-canary/0.log" Sep 15 12:43:35.377790 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:35.377756 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j7jz2_51cf3230-1af6-480a-9c83-6ad54f2013cf/kube-rbac-proxy/0.log" Sep 15 12:43:35.401152 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:35.401119 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j7jz2_51cf3230-1af6-480a-9c83-6ad54f2013cf/exporter/0.log" Sep 15 12:43:35.418425 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:35.418391 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j7jz2_51cf3230-1af6-480a-9c83-6ad54f2013cf/extractor/0.log" Sep 15 12:43:36.676060 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:36.676033 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n4f4n/perf-node-gather-daemonset-m4h8m" Sep 15 12:43:37.906883 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:37.906801 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jqns7_26665ca0-d0ee-48aa-958f-add728583758/server/0.log" Sep 15 12:43:38.052438 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:38.052407 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-422xq_87db7543-c2aa-4a47-9363-9fdd047dafa4/manager/0.log" Sep 15 12:43:38.183477 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:38.183381 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b747b9585-74lgg_4dc67243-5de9-4de1-bc50-df52da5facf8/manager/0.log" Sep 15 12:43:38.201966 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:38.201938 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b747b9585-74lgg_4dc67243-5de9-4de1-bc50-df52da5facf8/copy-manifests/0.log" Sep 15 12:43:39.405837 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:39.405728 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv_5906647c-59ec-4391-8389-8dcc624b1b1f/istio-proxy/0.log" Sep 15 12:43:39.719357 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:39.719330 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-86c794cf5f-vjdqv_6bf8bea2-da4b-468e-85a9-95c392401a04/manager/0.log" Sep 15 12:43:39.749191 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:39.749167 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-mhbtj_4c92c9b8-295a-45be-af60-2ec60b716fcd/openshift-lws-operator/0.log" Sep 15 12:43:40.257935 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:40.257904 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ccvgl_091727bd-6619-4ee6-8be2-718eec0832ec/discovery/0.log" Sep 15 12:43:40.306151 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:40.306119 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-nv8bv_c61cf7af-1e83-4674-b1b5-8795276fd46f/istio-proxy/0.log" Sep 15 12:43:40.311362 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:40.311334 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-nv8bv_c61cf7af-1e83-4674-b1b5-8795276fd46f/istio-proxy/1.log" Sep 15 12:43:40.366459 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:40.366427 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-clfp7_bd363dc1-4a24-4aed-83d0-d5bcbdedd04f/payload-processing/0.log" Sep 15 12:43:42.148970 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:42.148923 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6vgrv_5906647c-59ec-4391-8389-8dcc624b1b1f/istio-proxy/0.log" Sep 15 12:43:43.019014 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:43.018980 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ccvgl_091727bd-6619-4ee6-8be2-718eec0832ec/discovery/0.log" Sep 15 12:43:43.085862 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:43.085828 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-nv8bv_c61cf7af-1e83-4674-b1b5-8795276fd46f/istio-proxy/0.log" Sep 15 12:43:43.091375 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:43.091345 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-nv8bv_c61cf7af-1e83-4674-b1b5-8795276fd46f/istio-proxy/1.log" Sep 15 12:43:43.155106 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:43.155071 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-clfp7_bd363dc1-4a24-4aed-83d0-d5bcbdedd04f/payload-processing/0.log" Sep 15 12:43:46.159402 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:46.159373 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlbqb_a99cecea-5828-4a9d-a608-5a64fefb91aa/kube-multus-additional-cni-plugins/0.log" Sep 15 12:43:46.178908 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:46.178872 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlbqb_a99cecea-5828-4a9d-a608-5a64fefb91aa/egress-router-binary-copy/0.log" Sep 15 12:43:46.199366 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:46.199342 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlbqb_a99cecea-5828-4a9d-a608-5a64fefb91aa/cni-plugins/0.log" Sep 15 12:43:46.217595 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:46.217564 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlbqb_a99cecea-5828-4a9d-a608-5a64fefb91aa/bond-cni-plugin/0.log" Sep 15 12:43:46.238933 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:46.238897 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlbqb_a99cecea-5828-4a9d-a608-5a64fefb91aa/routeoverride-cni/0.log" Sep 15 12:43:46.257808 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:46.257781 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlbqb_a99cecea-5828-4a9d-a608-5a64fefb91aa/whereabouts-cni-bincopy/0.log" Sep 15 12:43:46.276901 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:46.276873 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mlbqb_a99cecea-5828-4a9d-a608-5a64fefb91aa/whereabouts-cni/0.log" Sep 15 12:43:46.483458 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:46.483427 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ksqcc_2143e3e2-6a05-4656-ac84-68b7e7189041/kube-multus/0.log" Sep 15 12:43:46.647066 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:46.647027 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s62bl_4a77b2b1-873f-4271-aabe-fcdff5647007/network-metrics-daemon/0.log" Sep 15 12:43:46.661356 ip-10-0-130-139 kubenswrapper[2582]: I0915 12:43:46.661328 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s62bl_4a77b2b1-873f-4271-aabe-fcdff5647007/kube-rbac-proxy/0.log"