Sep 08 07:02:36.396811 ip-10-0-142-34 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 08 07:02:36.396827 ip-10-0-142-34 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 08 07:02:36.396837 ip-10-0-142-34 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 08 07:02:36.397050 ip-10-0-142-34 systemd[1]: Failed to start Kubernetes Kubelet. Sep 08 07:02:46.455559 ip-10-0-142-34 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 08 07:02:46.455574 ip-10-0-142-34 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c2cc0cddb3ef43c7a481556a175edf75 -- Sep 08 07:04:45.208500 ip-10-0-142-34 systemd[1]: Starting Kubernetes Kubelet... Sep 08 07:04:45.584226 ip-10-0-142-34 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 07:04:45.584226 ip-10-0-142-34 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 08 07:04:45.584226 ip-10-0-142-34 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 07:04:45.584226 ip-10-0-142-34 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 08 07:04:45.584226 ip-10-0-142-34 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 07:04:45.584874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.584308 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 08 07:04:45.591622 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591599 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 07:04:45.591622 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591618 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 07:04:45.591622 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591622 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 07:04:45.591622 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591625 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 07:04:45.591622 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591630 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591633 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591636 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591639 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591642 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591644 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591649 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591652 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591655 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591658 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591661 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591663 2571 feature_gate.go:328] unrecognized feature gate: Example Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591666 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591669 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591672 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591674 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591677 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591679 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591682 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 07:04:45.591824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591685 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591687 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591690 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591693 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591695 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591698 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591700 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591703 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591708 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591711 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591713 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591716 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591719 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591721 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591724 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591727 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591730 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591732 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591735 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591738 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 07:04:45.592301 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591741 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591743 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591746 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591748 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591751 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591755 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591757 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591766 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591768 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591771 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591773 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591776 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591779 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591781 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591784 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591786 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591789 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591792 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591794 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591797 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 07:04:45.592833 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591800 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591802 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591805 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591807 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591810 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591813 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591816 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591819 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591821 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591824 2571 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591827 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591830 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591832 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591835 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591839 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591843 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591846 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591849 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591852 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 07:04:45.593323 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591855 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591863 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591866 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591871 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591875 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591878 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591881 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591884 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591887 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591895 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591898 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591901 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591903 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591906 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591909 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591912 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 07:04:45.593776 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.591914 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.593999 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594007 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594011 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594014 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594017 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594020 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594023 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594026 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594029 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594032 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594034 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594037 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594039 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594042 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594045 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594047 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594050 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594053 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594055 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 07:04:45.594166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594058 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594060 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594063 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594065 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594068 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594070 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594073 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594076 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594078 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594080 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594083 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594086 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594088 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594091 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594096 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594100 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594102 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594106 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594111 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 07:04:45.594669 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594114 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594116 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594119 2571 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594122 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594124 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594127 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594129 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594132 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594135 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594137 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594140 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594142 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594146 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594149 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594151 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594154 2571 feature_gate.go:328] unrecognized feature gate: Example Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594156 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594159 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594162 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594164 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 07:04:45.595167 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594167 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594170 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594172 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594175 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594178 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594180 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594183 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594187 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594189 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594192 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594195 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594197 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594200 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594204 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594208 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594228 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594231 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594234 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594236 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 07:04:45.595703 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594239 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594241 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594244 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594246 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594249 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594251 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594254 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594257 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594259 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594262 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594266 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594269 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594271 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594274 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594277 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594279 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594282 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594285 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594290 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 07:04:45.596166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594293 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594296 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594299 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594686 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594691 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594694 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594697 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594700 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594702 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594705 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594708 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594711 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594713 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594716 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594719 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594721 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594724 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594727 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594729 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 07:04:45.596641 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594733 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594735 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594738 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594741 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594743 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594746 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594748 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594755 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594757 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594760 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594762 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594766 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594769 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594772 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594775 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594777 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594780 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594783 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594785 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594788 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 07:04:45.597187 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594791 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594793 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594796 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594799 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594801 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594804 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594806 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594810 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594813 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594816 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594819 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594821 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594829 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594839 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594842 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594845 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594848 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594851 2571 feature_gate.go:328] unrecognized feature gate: Example Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594854 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 07:04:45.597695 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594856 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594859 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594862 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594865 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594870 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594873 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594876 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594878 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594881 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594884 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594886 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594889 2571 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594891 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594894 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594896 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594899 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594902 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594912 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594915 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594918 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 07:04:45.598161 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594920 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594923 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594925 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594928 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594930 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594932 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594935 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594944 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594946 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594949 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594954 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594956 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594959 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594961 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594964 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594966 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594970 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594975 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594978 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 07:04:45.598656 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594981 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594984 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594987 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594990 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.594993 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595065 2571 flags.go:64] FLAG: --address="0.0.0.0" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595101 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595107 2571 flags.go:64] FLAG: --anonymous-auth="true" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595112 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595116 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595119 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595123 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595127 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595131 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595134 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595137 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595141 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595144 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595147 2571 flags.go:64] FLAG: --cgroup-root="" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595150 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595153 2571 flags.go:64] FLAG: --client-ca-file="" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595156 2571 flags.go:64] FLAG: --cloud-provider="external" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595159 2571 flags.go:64] FLAG: --cluster-dns="[]" Sep 08 07:04:45.599117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595164 2571 flags.go:64] FLAG: --cluster-domain="" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595167 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595171 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595174 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595177 2571 flags.go:64] FLAG: --container-log-max-files="5" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595181 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595185 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595189 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595193 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595196 2571 flags.go:64] FLAG: --contention-profiling="false" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595199 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595201 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595205 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595207 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595224 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595227 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595230 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595233 2571 flags.go:64] FLAG: --enable-load-reader="false" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595236 2571 flags.go:64] FLAG: --enable-server="true" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595240 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595246 2571 flags.go:64] FLAG: --event-burst="100" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595249 2571 flags.go:64] FLAG: --event-qps="50" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595252 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595255 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 08 07:04:45.599681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595258 2571 flags.go:64] FLAG: --eviction-hard="" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595261 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595264 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595267 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595270 2571 flags.go:64] FLAG: --eviction-soft="" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595273 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595276 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595279 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595283 2571 flags.go:64] FLAG: --experimental-mounter-path="" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595286 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595289 2571 flags.go:64] FLAG: --fail-swap-on="true" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595292 2571 flags.go:64] FLAG: --feature-gates="" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595296 2571 flags.go:64] FLAG: --file-check-frequency="20s" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595299 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595302 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595306 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595309 2571 flags.go:64] FLAG: --healthz-port="10248" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595312 2571 flags.go:64] FLAG: --help="false" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595315 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-142-34.ec2.internal" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595318 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595321 2571 flags.go:64] FLAG: --http-check-frequency="20s" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595324 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595327 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 08 07:04:45.600293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595330 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595333 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595336 2571 flags.go:64] FLAG: --image-service-endpoint="" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595339 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595342 2571 flags.go:64] FLAG: --kube-api-burst="100" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595345 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595348 2571 flags.go:64] FLAG: --kube-api-qps="50" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595351 2571 flags.go:64] FLAG: --kube-reserved="" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595354 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595356 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595359 2571 flags.go:64] FLAG: --kubelet-cgroups="" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595362 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595365 2571 flags.go:64] FLAG: --lock-file="" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595367 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595370 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595373 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595378 2571 flags.go:64] FLAG: --log-json-split-stream="false" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595383 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595386 2571 flags.go:64] FLAG: --log-text-split-stream="false" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595389 2571 flags.go:64] FLAG: --logging-format="text" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595391 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595395 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595398 2571 flags.go:64] FLAG: --manifest-url="" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595400 2571 flags.go:64] FLAG: --manifest-url-header="" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595406 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 08 07:04:45.600908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595409 2571 flags.go:64] FLAG: --max-open-files="1000000" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595413 2571 flags.go:64] FLAG: --max-pods="110" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595416 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595418 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595421 2571 flags.go:64] FLAG: --memory-manager-policy="None" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595424 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595427 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595430 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595433 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595440 2571 flags.go:64] FLAG: --node-status-max-images="50" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595443 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595446 2571 flags.go:64] FLAG: --oom-score-adj="-999" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595450 2571 flags.go:64] FLAG: --pod-cidr="" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595453 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595458 2571 flags.go:64] FLAG: --pod-manifest-path="" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595461 2571 flags.go:64] FLAG: --pod-max-pids="-1" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595464 2571 flags.go:64] FLAG: --pods-per-core="0" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595467 2571 flags.go:64] FLAG: --port="10250" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595470 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595473 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0af9080488cf83da0" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595476 2571 flags.go:64] FLAG: --qos-reserved="" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595479 2571 flags.go:64] FLAG: --read-only-port="10255" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595482 2571 flags.go:64] FLAG: --register-node="true" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595486 2571 flags.go:64] FLAG: --register-with-taints="" Sep 08 07:04:45.601581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595491 2571 flags.go:64] FLAG: --registry-burst="10" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595494 2571 flags.go:64] FLAG: --registry-qps="5" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595497 2571 flags.go:64] FLAG: --reserved-cpus="" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595500 2571 flags.go:64] FLAG: --reserved-memory="" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595503 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595506 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595509 2571 flags.go:64] FLAG: --rotate-certificates="false" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595512 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595516 2571 flags.go:64] FLAG: --runonce="false" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595519 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595522 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595525 2571 flags.go:64] FLAG: --seccomp-default="false" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595528 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595531 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595534 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595538 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595541 2571 flags.go:64] FLAG: --storage-driver-password="root" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595544 2571 flags.go:64] FLAG: --storage-driver-secure="false" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595547 2571 flags.go:64] FLAG: --storage-driver-table="stats" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595550 2571 flags.go:64] FLAG: --storage-driver-user="root" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595552 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595556 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595559 2571 flags.go:64] FLAG: --system-cgroups="" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595561 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595567 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 08 07:04:45.602183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595569 2571 flags.go:64] FLAG: --tls-cert-file="" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595572 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595576 2571 flags.go:64] FLAG: --tls-min-version="" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595579 2571 flags.go:64] FLAG: --tls-private-key-file="" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595581 2571 flags.go:64] FLAG: --topology-manager-policy="none" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595584 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595587 2571 flags.go:64] FLAG: --topology-manager-scope="container" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595592 2571 flags.go:64] FLAG: --v="2" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595595 2571 flags.go:64] FLAG: --version="false" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595600 2571 flags.go:64] FLAG: --vmodule="" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595604 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.595607 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595718 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595722 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595726 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595730 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595733 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595736 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595738 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595741 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595744 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595747 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 07:04:45.602810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595749 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595752 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595755 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595757 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595760 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595763 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595765 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595768 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595771 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595774 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595776 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595779 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595781 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595784 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595787 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595790 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595793 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595796 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595798 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595802 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 07:04:45.603378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595810 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595813 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595816 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595818 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595821 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595825 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595828 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595830 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595833 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595836 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595839 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595841 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595844 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595846 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595849 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595852 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595854 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595857 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595860 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 07:04:45.603871 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595862 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595865 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595868 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595870 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595874 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595877 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595880 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595884 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595886 2571 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595890 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595894 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595897 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595900 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595903 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595906 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595909 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595911 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595914 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595918 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 07:04:45.604342 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595921 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595923 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595926 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595929 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595932 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595934 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595937 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595940 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595942 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595945 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595948 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595951 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595953 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595956 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595959 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595961 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595972 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595975 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595978 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595981 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 07:04:45.604845 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595983 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595986 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595988 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595991 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595994 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.595996 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.596000 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.596002 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.596005 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.596008 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.596010 2571 feature_gate.go:328] unrecognized feature gate: Example Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.596536 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.603745 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.603763 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.603782 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 08 07:04:45.605419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.603791 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603858 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603862 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603866 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603869 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603872 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603875 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603878 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603881 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603884 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603887 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603889 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603892 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603894 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603897 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603900 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603902 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603905 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603907 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 07:04:45.605810 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603910 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603913 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603916 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603918 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603922 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603926 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603929 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603932 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603935 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603937 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603941 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603952 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603956 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603958 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603961 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603964 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603967 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603969 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603972 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603975 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 07:04:45.606284 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603978 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603981 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603984 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603986 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603989 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603993 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603996 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.603999 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604002 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604005 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604007 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604010 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604012 2571 feature_gate.go:328] unrecognized feature gate: Example Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604015 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604018 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604020 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604023 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604025 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604028 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 07:04:45.606760 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604030 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604034 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604037 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604040 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604043 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604052 2571 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604054 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604058 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604061 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604063 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604066 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604069 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604072 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604075 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604077 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604080 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604083 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604085 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604088 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604091 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 07:04:45.607228 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604093 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604096 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604098 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604101 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604104 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604106 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604109 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604112 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604114 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604117 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604119 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604122 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604124 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604126 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604129 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604132 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604135 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604137 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604146 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604148 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 07:04:45.607727 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604151 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604153 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.604158 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604337 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604343 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604346 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604350 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604353 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604356 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604359 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604362 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604365 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604369 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604373 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 07:04:45.608253 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604377 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604380 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604383 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604386 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604389 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604391 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604394 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604397 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604399 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604402 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604404 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604407 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604409 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604412 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604415 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604417 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604420 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604429 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604432 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 07:04:45.608591 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604434 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604437 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604440 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604442 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604445 2571 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604448 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604451 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604453 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604456 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604460 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604463 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604467 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604470 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604472 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604475 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604478 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604480 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604483 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604486 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604488 2571 feature_gate.go:328] unrecognized feature gate: Example Sep 08 07:04:45.609053 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604491 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604493 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604496 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604498 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604501 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604504 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604506 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604509 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604512 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604514 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604517 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604528 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604531 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604533 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604536 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604539 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604541 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604543 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604546 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604549 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 07:04:45.609590 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604551 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604554 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604557 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604560 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604562 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604565 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604567 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604570 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604572 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604575 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604577 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604580 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604582 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604585 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604588 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604590 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604593 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604597 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604600 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604602 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 07:04:45.610066 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604605 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604608 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604610 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604613 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604621 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604624 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604626 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604629 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:45.604631 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.604636 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.604769 2571 server.go:962] "Client rotation is on, will bootstrap in background" Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.607776 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.608526 2571 server.go:1019] "Starting client certificate rotation" Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.608655 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 07:04:45.610565 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.609648 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 07:04:45.610960 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.610945 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 07:04:45.612477 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.612457 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 07:04:45.622659 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.622631 2571 log.go:25] "Validated CRI v1 runtime API" Sep 08 07:04:45.626444 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.626421 2571 log.go:25] "Validated CRI v1 image API" Sep 08 07:04:45.627371 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.627354 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 08 07:04:45.630466 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.630449 2571 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ae660415-ac6a-4b0d-ac37-2d83a1ed8936:/dev/nvme0n1p4 b447c70f-9a11-4f49-ba14-a8e64a5c24bc:/dev/nvme0n1p3] Sep 08 07:04:45.630525 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.630465 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 08 07:04:45.635642 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.635531 2571 manager.go:216] Machine: {Timestamp:2026-09-08 07:04:45.634101544 +0000 UTC m=+0.312064671 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095398 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27404562f10205195f91943cde6ada SystemUUID:ec274045-62f1-0205-195f-91943cde6ada BootID:c2cc0cdd-b3ef-43c7-a481-556a175edf75 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d8:d4:ca:fd:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d8:d4:ca:fd:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:21:86:c9:04:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 08 07:04:45.635642 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.635638 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 08 07:04:45.635765 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.635721 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 08 07:04:45.638364 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.638348 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 07:04:45.638603 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.638578 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 08 07:04:45.638762 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.638611 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-34.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 08 07:04:45.639799 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.639787 2571 topology_manager.go:138] "Creating topology manager with none policy" Sep 08 07:04:45.639799 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.639801 2571 container_manager_linux.go:309] "Creating device plugin manager" Sep 08 07:04:45.639874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.639809 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 07:04:45.640415 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.640405 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 07:04:45.641639 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.641630 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 08 07:04:45.644453 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.644443 2571 state_mem.go:36] "Initialized new in-memory state store" Sep 08 07:04:45.644576 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.644567 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 08 07:04:45.644610 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.644596 2571 kubelet.go:486] "Attempting to sync node with API server" Sep 08 07:04:45.644610 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.644605 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 08 07:04:45.644662 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.644623 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 08 07:04:45.644662 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.644633 2571 kubelet.go:398] "Adding apiserver pod source" Sep 08 07:04:45.644662 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.644641 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 08 07:04:45.645036 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.645021 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 07:04:45.645098 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.645042 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 07:04:45.647390 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.647374 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 08 07:04:45.651271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.651258 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 08 07:04:45.651336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.651288 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 08 07:04:45.652024 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652010 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 08 07:04:45.652069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652037 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 08 07:04:45.652069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652046 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 08 07:04:45.652069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652055 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 08 07:04:45.652069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652067 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 08 07:04:45.652177 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652078 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 08 07:04:45.652177 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652097 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 08 07:04:45.652177 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652112 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 08 07:04:45.652177 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652122 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 08 07:04:45.652177 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652131 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 08 07:04:45.652177 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652143 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 08 07:04:45.652177 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652151 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 08 07:04:45.652857 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652846 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 08 07:04:45.652857 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.652858 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 08 07:04:45.653299 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.653274 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-34.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 08 07:04:45.653436 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.653389 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 08 07:04:45.656472 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.656457 2571 server.go:1268] "Started kubelet" Sep 08 07:04:45.656592 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.656562 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 08 07:04:45.656628 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.656556 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 08 07:04:45.656675 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.656664 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 08 07:04:45.657632 ip-10-0-142-34 systemd[1]: Started Kubernetes Kubelet. Sep 08 07:04:45.658484 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.658289 2571 server.go:310] "Adding debug handlers to kubelet server" Sep 08 07:04:45.660045 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.660027 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 08 07:04:45.660559 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.660521 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-grt6d" Sep 08 07:04:45.662597 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.662577 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 08 07:04:45.663311 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.662380 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-34.ec2.internal.18d34695ac9e7763 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-34.ec2.internal,UID:ip-10-0-142-34.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-34.ec2.internal,},FirstTimestamp:2026-09-08 07:04:45.656356707 +0000 UTC m=+0.334319834,LastTimestamp:2026-09-08 07:04:45.656356707 +0000 UTC m=+0.334319834,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-34.ec2.internal,}" Sep 08 07:04:45.664661 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.664648 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 08 07:04:45.664711 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.664667 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 08 07:04:45.664756 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.664729 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:45.665450 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665430 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 08 07:04:45.665691 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665599 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 08 07:04:45.665691 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665615 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 08 07:04:45.665819 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665684 2571 reconstruct.go:97] "Volume reconstruction finished" Sep 08 07:04:45.665819 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665711 2571 reconciler.go:29] "Reconciler: start to sync state" Sep 08 07:04:45.665819 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665737 2571 factory.go:153] Registering CRI-O factory Sep 08 07:04:45.665819 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665769 2571 factory.go:223] Registration of the crio container factory successfully Sep 08 07:04:45.665819 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665816 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 08 07:04:45.666069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665825 2571 factory.go:55] Registering systemd factory Sep 08 07:04:45.666069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665832 2571 factory.go:223] Registration of the systemd container factory successfully Sep 08 07:04:45.666069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665848 2571 factory.go:103] Registering Raw factory Sep 08 07:04:45.666069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.665861 2571 manager.go:1196] Started watching for new ooms in manager Sep 08 07:04:45.667831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.667812 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-grt6d" Sep 08 07:04:45.668549 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.668528 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-34.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 08 07:04:45.668645 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.668608 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 08 07:04:45.668727 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.668708 2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-34.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 08 07:04:45.668805 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.668794 2571 manager.go:349] Starting recovery of all containers Sep 08 07:04:45.678703 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.678685 2571 manager.go:354] Recovery completed Sep 08 07:04:45.682889 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.682778 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 07:04:45.684717 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.684593 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientMemory" Sep 08 07:04:45.684717 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.684628 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 07:04:45.684717 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.684645 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientPID" Sep 08 07:04:45.685312 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.685293 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 08 07:04:45.685312 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.685306 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 07:04:45.685312 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.685310 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 08 07:04:45.685459 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.685328 2571 state_mem.go:36] "Initialized new in-memory state store" Sep 08 07:04:45.686031 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.686017 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientMemory" Sep 08 07:04:45.686095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.686043 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 07:04:45.686095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.686059 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientPID" Sep 08 07:04:45.686430 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.686419 2571 policy_none.go:49] "None policy: Start" Sep 08 07:04:45.686462 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.686435 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 08 07:04:45.686462 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.686444 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 08 07:04:45.687496 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.687481 2571 policy_none.go:47] "Start" Sep 08 07:04:45.717929 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.717913 2571 manager.go:337] "Starting Device Plugin manager" Sep 08 07:04:45.732744 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.717951 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 08 07:04:45.732744 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.717960 2571 server.go:87] "Starting device plugin registration server" Sep 08 07:04:45.732744 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.718333 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 08 07:04:45.732744 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.718344 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 08 07:04:45.732744 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.718440 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 08 07:04:45.732744 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.718518 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 08 07:04:45.732744 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.718529 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 08 07:04:45.732744 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.719098 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 08 07:04:45.732744 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.719145 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:45.801088 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.801046 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 08 07:04:45.802307 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.802289 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 08 07:04:45.802371 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.802355 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 08 07:04:45.802416 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.802392 2571 kubelet.go:2448] "Starting kubelet main sync loop" Sep 08 07:04:45.802507 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.802486 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 08 07:04:45.804468 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.804450 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 07:04:45.819267 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.819237 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 07:04:45.820104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.820086 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientMemory" Sep 08 07:04:45.820180 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.820117 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 07:04:45.820180 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.820127 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientPID" Sep 08 07:04:45.820180 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.820149 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-34.ec2.internal" Sep 08 07:04:45.828733 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.828716 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-34.ec2.internal" Sep 08 07:04:45.828831 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.828740 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-34.ec2.internal\": node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:45.841813 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.841743 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:45.903313 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.903281 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal"] Sep 08 07:04:45.903447 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.903367 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 07:04:45.904304 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.904281 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientMemory" Sep 08 07:04:45.904421 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.904314 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 07:04:45.904421 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.904326 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientPID" Sep 08 07:04:45.905668 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.905656 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 07:04:45.905802 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.905787 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" Sep 08 07:04:45.905837 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.905818 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 07:04:45.906399 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.906383 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientMemory" Sep 08 07:04:45.906476 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.906411 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 07:04:45.906476 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.906420 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientPID" Sep 08 07:04:45.906476 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.906382 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientMemory" Sep 08 07:04:45.906562 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.906489 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 07:04:45.906562 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.906500 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientPID" Sep 08 07:04:45.907599 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.907584 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal" Sep 08 07:04:45.907660 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.907615 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 07:04:45.908258 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.908242 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientMemory" Sep 08 07:04:45.908329 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.908269 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 07:04:45.908329 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:45.908282 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeHasSufficientPID" Sep 08 07:04:45.937855 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.937832 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-34.ec2.internal\" not found" node="ip-10-0-142-34.ec2.internal" Sep 08 07:04:45.942118 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.942101 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:45.942344 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:45.942328 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-34.ec2.internal\" not found" node="ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.042659 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:46.042607 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:46.066913 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.066885 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6b5baafa68872d0e1a89c3b7fe397e3a-config\") pod \"kube-apiserver-proxy-ip-10-0-142-34.ec2.internal\" (UID: \"6b5baafa68872d0e1a89c3b7fe397e3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.066990 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.066917 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e13a78121acfe93071c1c9eb406a02ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal\" (UID: \"e13a78121acfe93071c1c9eb406a02ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.066990 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.066937 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e13a78121acfe93071c1c9eb406a02ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal\" (UID: \"e13a78121acfe93071c1c9eb406a02ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.143167 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:46.143105 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:46.167299 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.167271 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e13a78121acfe93071c1c9eb406a02ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal\" (UID: \"e13a78121acfe93071c1c9eb406a02ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.167384 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.167303 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6b5baafa68872d0e1a89c3b7fe397e3a-config\") pod \"kube-apiserver-proxy-ip-10-0-142-34.ec2.internal\" (UID: \"6b5baafa68872d0e1a89c3b7fe397e3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.167384 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.167324 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e13a78121acfe93071c1c9eb406a02ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal\" (UID: \"e13a78121acfe93071c1c9eb406a02ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.167384 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.167371 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e13a78121acfe93071c1c9eb406a02ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal\" (UID: \"e13a78121acfe93071c1c9eb406a02ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.167482 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.167382 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6b5baafa68872d0e1a89c3b7fe397e3a-config\") pod \"kube-apiserver-proxy-ip-10-0-142-34.ec2.internal\" (UID: \"6b5baafa68872d0e1a89c3b7fe397e3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.167482 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.167359 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e13a78121acfe93071c1c9eb406a02ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal\" (UID: \"e13a78121acfe93071c1c9eb406a02ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.245421 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:46.245371 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:46.245492 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.245430 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.245492 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.245484 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" Sep 08 07:04:46.346493 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:46.346442 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:46.452642 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:46.452551 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:46.472902 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.472873 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 07:04:46.553087 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:46.553040 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:46.610154 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.610117 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 08 07:04:46.610661 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.610362 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 07:04:46.610661 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.610372 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 07:04:46.654010 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:46.653970 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:46.669521 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.669490 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 06:59:46 +0000 UTC" deadline="2028-06-16 19:34:53.389545348 +0000 UTC" Sep 08 07:04:46.669521 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.669518 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15540h30m6.720030422s" Sep 08 07:04:46.754658 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:46.754627 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:46.852499 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:46.852462 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13a78121acfe93071c1c9eb406a02ec.slice/crio-35eaaebdd24c6b6198bc606e22e96ecc875e2509ad2f2cbd73c564331baddd0c WatchSource:0}: Error finding container 35eaaebdd24c6b6198bc606e22e96ecc875e2509ad2f2cbd73c564331baddd0c: Status 404 returned error can't find the container with id 35eaaebdd24c6b6198bc606e22e96ecc875e2509ad2f2cbd73c564331baddd0c Sep 08 07:04:46.852751 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:46.852724 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5baafa68872d0e1a89c3b7fe397e3a.slice/crio-cde98212493381c3372e54d30a0492b72f429223d2a510661dbfd85d1d790640 WatchSource:0}: Error finding container cde98212493381c3372e54d30a0492b72f429223d2a510661dbfd85d1d790640: Status 404 returned error can't find the container with id cde98212493381c3372e54d30a0492b72f429223d2a510661dbfd85d1d790640 Sep 08 07:04:46.854931 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:46.854912 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:46.858893 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:46.858872 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 07:04:46.957642 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:46.957593 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-34.ec2.internal\" not found" Sep 08 07:04:47.009786 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.009717 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 07:04:47.064960 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.064933 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" Sep 08 07:04:47.076468 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.076446 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 07:04:47.079470 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.079455 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal" Sep 08 07:04:47.090193 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.090174 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 07:04:47.202605 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.202573 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 07:04:47.541967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.541933 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 07:04:47.646459 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.646189 2571 apiserver.go:52] "Watching apiserver" Sep 08 07:04:47.654200 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.654033 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 08 07:04:47.654552 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.654517 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-988kt","openshift-image-registry/node-ca-drmx7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal","openshift-multus/multus-678bx","openshift-multus/multus-additional-cni-plugins-dtfw9","kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal","openshift-cluster-node-tuning-operator/tuned-hlpl7","openshift-multus/network-metrics-daemon-8k4kq","openshift-network-diagnostics/network-check-target-zts7r","openshift-network-operator/iptables-alerter-b4q94","openshift-ovn-kubernetes/ovnkube-node-wsrt2","kube-system/global-pull-secret-syncer-gj26q","kube-system/konnectivity-agent-ptdrw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh"] Sep 08 07:04:47.656798 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.656286 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.657898 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.657393 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:04:47.660796 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.658619 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.660796 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.660206 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 08 07:04:47.660796 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.660279 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-678bx" Sep 08 07:04:47.660796 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.660514 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 08 07:04:47.662049 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.661191 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wmv2r\"" Sep 08 07:04:47.662049 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.661305 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 08 07:04:47.662049 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.661423 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 08 07:04:47.662049 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.661468 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n5zqc\"" Sep 08 07:04:47.662049 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.661735 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 08 07:04:47.662049 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.661964 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 08 07:04:47.664412 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.664022 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kjbvg\"" Sep 08 07:04:47.666872 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.665998 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 08 07:04:47.666872 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.666736 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 08 07:04:47.668681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.667106 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bv9m8\"" Sep 08 07:04:47.668681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.667559 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 08 07:04:47.668681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.667781 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 08 07:04:47.668681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.667901 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 08 07:04:47.668681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.668575 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 08 07:04:47.670874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.670852 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.675326 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.674672 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:47.675326 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.674771 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:04:47.675326 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.674951 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 08 07:04:47.675326 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.675048 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 08 07:04:47.675326 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.675277 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n8wtf\"" Sep 08 07:04:47.676917 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.676230 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:47.676917 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.676324 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:04:47.677708 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.677686 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:47.678339 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678286 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-modprobe-d\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.678339 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678322 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-sys\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.678485 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678348 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-var-lib-kubelet\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.678485 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678373 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-device-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.678485 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678416 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.678485 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678455 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-cni-binary-copy\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.678701 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678500 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-run-netns\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.678701 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678546 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73d46ded-8a5c-46ba-abda-1359a34cd12c-cni-binary-copy\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.678701 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678592 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-systemd\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.678701 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678632 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-cni-dir\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.678701 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678659 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-var-lib-kubelet\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.678701 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678692 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-hostroot\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.679007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678741 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-conf-dir\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.679007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678770 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs2qh\" (UniqueName: \"kubernetes.io/projected/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-kube-api-access-vs2qh\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.679007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678797 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:47.679007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678851 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-tuned\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.679007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678878 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltnrf\" (UniqueName: \"kubernetes.io/projected/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-kube-api-access-ltnrf\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.679007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678903 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-system-cni-dir\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.679007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678927 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-socket-dir-parent\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.679007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.678952 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-var-lib-cni-multus\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679009 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73d46ded-8a5c-46ba-abda-1359a34cd12c-system-cni-dir\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679040 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73d46ded-8a5c-46ba-abda-1359a34cd12c-cnibin\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679072 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-run\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679100 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-registration-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679132 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-etc-kubernetes\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679164 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/73d46ded-8a5c-46ba-abda-1359a34cd12c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679201 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m2cz\" (UniqueName: \"kubernetes.io/projected/73d46ded-8a5c-46ba-abda-1359a34cd12c-kube-api-access-9m2cz\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679239 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbsjf\" (UniqueName: \"kubernetes.io/projected/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-kube-api-access-pbsjf\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679264 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-sysctl-conf\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679286 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-host\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679309 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-sys-fs\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679357 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-var-lib-cni-bin\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.679419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679392 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-kubernetes\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679427 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-tmp\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679452 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-run-k8s-cni-cncf-io\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679480 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-run-multus-certs\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679506 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/73d46ded-8a5c-46ba-abda-1359a34cd12c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679529 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-sysctl-d\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679541 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679552 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-lib-modules\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679585 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0d3cb711-52d1-4457-a3e2-ce1efbefbfe1-agent-certs\") pod \"konnectivity-agent-ptdrw\" (UID: \"0d3cb711-52d1-4457-a3e2-ce1efbefbfe1\") " pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679608 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0d3cb711-52d1-4457-a3e2-ce1efbefbfe1-konnectivity-ca\") pod \"konnectivity-agent-ptdrw\" (UID: \"0d3cb711-52d1-4457-a3e2-ce1efbefbfe1\") " pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679630 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679655 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73d46ded-8a5c-46ba-abda-1359a34cd12c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679679 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-sysconfig\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679704 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-socket-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679728 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-cnibin\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679752 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-daemon-config\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.680095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.679821 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8cbp\" (UniqueName: \"kubernetes.io/projected/b282970a-79b3-42c5-8200-192bf1b0ac43-kube-api-access-n8cbp\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:47.681030 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.680184 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 08 07:04:47.681030 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.680260 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7rwxn\"" Sep 08 07:04:47.681030 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.680547 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 08 07:04:47.681030 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.680849 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:47.681439 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.681416 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 07:04:47.682080 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.682061 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 08 07:04:47.682161 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.682131 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 08 07:04:47.682237 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.682061 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7g8gw\"" Sep 08 07:04:47.682237 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.682223 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 08 07:04:47.682731 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.682714 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.683809 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.683787 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 08 07:04:47.683924 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.683832 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 08 07:04:47.684114 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.684097 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n8jxb\"" Sep 08 07:04:47.684220 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.684201 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:47.684458 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.684432 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:04:47.684969 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.684215 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 08 07:04:47.685804 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.685477 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 08 07:04:47.685804 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.685490 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 08 07:04:47.685804 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.685538 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-78gfd\"" Sep 08 07:04:47.687711 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.686801 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 08 07:04:47.687711 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.686877 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 08 07:04:47.687711 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.687190 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 08 07:04:47.687711 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.687337 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 08 07:04:47.705539 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.705514 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6jvv6" Sep 08 07:04:47.712063 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.712040 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6jvv6" Sep 08 07:04:47.769742 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.767885 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 08 07:04:47.780441 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780415 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-run-openvswitch\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.780568 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780445 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-run-ovn\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.780568 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780464 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-cni-netd\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.780568 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780494 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-tmp\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.780672 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780551 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-run-multus-certs\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.780672 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780605 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ace1cd1-322a-41ec-82df-53bc6914bd50-tmp-dir\") pod \"node-resolver-988kt\" (UID: \"4ace1cd1-322a-41ec-82df-53bc6914bd50\") " pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:47.780672 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780635 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-run-ovn-kubernetes\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.780672 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780664 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-sysctl-d\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.780822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780689 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-lib-modules\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.780822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780713 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-cni-bin\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.780822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780738 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-socket-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.780822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780762 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-cnibin\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.780822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780787 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-daemon-config\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.780822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780817 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-sys\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780840 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-device-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780864 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-cni-binary-copy\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780888 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-run-netns\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780913 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73d46ded-8a5c-46ba-abda-1359a34cd12c-cni-binary-copy\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780940 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-kubelet-config\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780965 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-systemd\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.780991 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-conf-dir\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781016 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/724e0b43-1d19-449a-8dcb-a3908b474a9f-env-overrides\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781042 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltnrf\" (UniqueName: \"kubernetes.io/projected/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-kube-api-access-ltnrf\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781069 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73d46ded-8a5c-46ba-abda-1359a34cd12c-system-cni-dir\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781085 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 08 07:04:47.781104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781094 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-run-systemd\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.781534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781120 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-log-socket\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.781534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781148 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kntkk\" (UniqueName: \"kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk\") pod \"network-check-target-zts7r\" (UID: \"433a278e-8a9d-44ad-972c-f36e39dd23d0\") " pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:47.781534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781173 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-run\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.781534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781224 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-registration-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.781534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781252 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-etc-kubernetes\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.781534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781292 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-node-log\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.781534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781320 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbsjf\" (UniqueName: \"kubernetes.io/projected/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-kube-api-access-pbsjf\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.781534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781347 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-sysctl-conf\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.781534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781373 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-var-lib-cni-bin\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.781534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781446 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-var-lib-cni-bin\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.781911 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781579 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-systemd\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.781911 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781623 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-conf-dir\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.781911 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781881 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73d46ded-8a5c-46ba-abda-1359a34cd12c-cni-binary-copy\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.781911 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781901 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73d46ded-8a5c-46ba-abda-1359a34cd12c-system-cni-dir\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.782078 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781940 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b05cd8e7-67da-40fe-a01d-d766c7d73ef9-host\") pod \"node-ca-drmx7\" (UID: \"b05cd8e7-67da-40fe-a01d-d766c7d73ef9\") " pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:47.782078 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.781969 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa3811e3-5018-432a-8a05-49fabb23b1a3-host-slash\") pod \"iptables-alerter-b4q94\" (UID: \"aa3811e3-5018-432a-8a05-49fabb23b1a3\") " pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:47.782078 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782067 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-run\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.782236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782107 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-sysctl-conf\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.782236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782117 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-registration-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.782236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782153 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/724e0b43-1d19-449a-8dcb-a3908b474a9f-ovn-node-metrics-cert\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.782236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782188 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-lib-modules\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.782236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782191 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xws2p\" (UniqueName: \"kubernetes.io/projected/724e0b43-1d19-449a-8dcb-a3908b474a9f-kube-api-access-xws2p\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.782236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782199 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-run-multus-certs\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.782236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782219 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-dbus\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:47.782236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782225 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-sys\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782250 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-kubernetes\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782282 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-device-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782338 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-run-k8s-cni-cncf-io\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782355 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-socket-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782367 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/73d46ded-8a5c-46ba-abda-1359a34cd12c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782382 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-kubernetes\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782394 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkthl\" (UniqueName: \"kubernetes.io/projected/aa3811e3-5018-432a-8a05-49fabb23b1a3-kube-api-access-zkthl\") pod \"iptables-alerter-b4q94\" (UID: \"aa3811e3-5018-432a-8a05-49fabb23b1a3\") " pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782413 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-etc-kubernetes\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782421 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-var-lib-openvswitch\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782461 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-run-k8s-cni-cncf-io\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782461 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-sysctl-d\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782486 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-run-netns\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782504 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-cnibin\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.782646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782499 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782951 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/73d46ded-8a5c-46ba-abda-1359a34cd12c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.782996 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0d3cb711-52d1-4457-a3e2-ce1efbefbfe1-agent-certs\") pod \"konnectivity-agent-ptdrw\" (UID: \"0d3cb711-52d1-4457-a3e2-ce1efbefbfe1\") " pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783017 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0d3cb711-52d1-4457-a3e2-ce1efbefbfe1-konnectivity-ca\") pod \"konnectivity-agent-ptdrw\" (UID: \"0d3cb711-52d1-4457-a3e2-ce1efbefbfe1\") " pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783069 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783073 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-daemon-config\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783099 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-cni-binary-copy\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783092 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73d46ded-8a5c-46ba-abda-1359a34cd12c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783156 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xzxn\" (UniqueName: \"kubernetes.io/projected/b05cd8e7-67da-40fe-a01d-d766c7d73ef9-kube-api-access-9xzxn\") pod \"node-ca-drmx7\" (UID: \"b05cd8e7-67da-40fe-a01d-d766c7d73ef9\") " pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783181 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-kubelet\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783210 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-sysconfig\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783256 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8cbp\" (UniqueName: \"kubernetes.io/projected/b282970a-79b3-42c5-8200-192bf1b0ac43-kube-api-access-n8cbp\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783338 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-slash\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783363 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783385 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkx5z\" (UniqueName: \"kubernetes.io/projected/4ace1cd1-322a-41ec-82df-53bc6914bd50-kube-api-access-tkx5z\") pod \"node-resolver-988kt\" (UID: \"4ace1cd1-322a-41ec-82df-53bc6914bd50\") " pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:47.783394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783402 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-modprobe-d\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783405 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783420 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-var-lib-kubelet\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783443 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783506 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/724e0b43-1d19-449a-8dcb-a3908b474a9f-ovnkube-script-lib\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783537 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-cni-dir\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783559 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-var-lib-kubelet\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783580 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-hostroot\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783596 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs2qh\" (UniqueName: \"kubernetes.io/projected/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-kube-api-access-vs2qh\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783613 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b05cd8e7-67da-40fe-a01d-d766c7d73ef9-serviceca\") pod \"node-ca-drmx7\" (UID: \"b05cd8e7-67da-40fe-a01d-d766c7d73ef9\") " pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783612 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0d3cb711-52d1-4457-a3e2-ce1efbefbfe1-konnectivity-ca\") pod \"konnectivity-agent-ptdrw\" (UID: \"0d3cb711-52d1-4457-a3e2-ce1efbefbfe1\") " pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783631 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783657 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-tuned\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783706 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-modprobe-d\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783734 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-system-cni-dir\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783744 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73d46ded-8a5c-46ba-abda-1359a34cd12c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783758 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-socket-dir-parent\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.784135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783779 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783790 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-var-lib-cni-multus\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783819 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73d46ded-8a5c-46ba-abda-1359a34cd12c-cnibin\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783829 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-sysconfig\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783847 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m2cz\" (UniqueName: \"kubernetes.io/projected/73d46ded-8a5c-46ba-abda-1359a34cd12c-kube-api-access-9m2cz\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783880 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ace1cd1-322a-41ec-82df-53bc6914bd50-hosts-file\") pod \"node-resolver-988kt\" (UID: \"4ace1cd1-322a-41ec-82df-53bc6914bd50\") " pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783903 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/73d46ded-8a5c-46ba-abda-1359a34cd12c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.783909 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783925 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aa3811e3-5018-432a-8a05-49fabb23b1a3-iptables-alerter-script\") pod \"iptables-alerter-b4q94\" (UID: \"aa3811e3-5018-432a-8a05-49fabb23b1a3\") " pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.783945 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-run-netns\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.783975 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs podName:b282970a-79b3-42c5-8200-192bf1b0ac43 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:48.283953172 +0000 UTC m=+2.961916271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs") pod "network-metrics-daemon-8k4kq" (UID: "b282970a-79b3-42c5-8200-192bf1b0ac43") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784036 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/724e0b43-1d19-449a-8dcb-a3908b474a9f-ovnkube-config\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784068 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-host\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784098 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-sys-fs\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784116 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-systemd-units\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784121 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-system-cni-dir\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784133 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-etc-openvswitch\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.784967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784205 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-socket-dir-parent\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.785881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784220 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-var-lib-kubelet\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.785881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784237 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-host-var-lib-cni-multus\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.785881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784269 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73d46ded-8a5c-46ba-abda-1359a34cd12c-cnibin\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.785881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784276 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-var-lib-kubelet\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.785881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784355 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-multus-cni-dir\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.785881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784510 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-host\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.785881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784542 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-sys-fs\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.785881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784575 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-hostroot\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.785881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.784908 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/73d46ded-8a5c-46ba-abda-1359a34cd12c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.785881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.785518 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-tmp\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.786990 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.786972 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-etc-tuned\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.786990 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.786985 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0d3cb711-52d1-4457-a3e2-ce1efbefbfe1-agent-certs\") pod \"konnectivity-agent-ptdrw\" (UID: \"0d3cb711-52d1-4457-a3e2-ce1efbefbfe1\") " pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:04:47.793238 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.793178 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8cbp\" (UniqueName: \"kubernetes.io/projected/b282970a-79b3-42c5-8200-192bf1b0ac43-kube-api-access-n8cbp\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:47.793238 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.793206 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltnrf\" (UniqueName: \"kubernetes.io/projected/ea14dea9-8914-47a0-9ee9-e6f3da515e8d-kube-api-access-ltnrf\") pod \"tuned-hlpl7\" (UID: \"ea14dea9-8914-47a0-9ee9-e6f3da515e8d\") " pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.794169 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.794148 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m2cz\" (UniqueName: \"kubernetes.io/projected/73d46ded-8a5c-46ba-abda-1359a34cd12c-kube-api-access-9m2cz\") pod \"multus-additional-cni-plugins-dtfw9\" (UID: \"73d46ded-8a5c-46ba-abda-1359a34cd12c\") " pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:47.794376 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.794349 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbsjf\" (UniqueName: \"kubernetes.io/projected/2bc7868f-cf26-41b9-b221-a5d5563e6cc2-kube-api-access-pbsjf\") pod \"aws-ebs-csi-driver-node-p4twh\" (UID: \"2bc7868f-cf26-41b9-b221-a5d5563e6cc2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:47.797517 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.797499 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs2qh\" (UniqueName: \"kubernetes.io/projected/18c085d6-4085-42a9-bf4c-ad13ad01f7e6-kube-api-access-vs2qh\") pod \"multus-678bx\" (UID: \"18c085d6-4085-42a9-bf4c-ad13ad01f7e6\") " pod="openshift-multus/multus-678bx" Sep 08 07:04:47.806721 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.806678 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" event={"ID":"e13a78121acfe93071c1c9eb406a02ec","Type":"ContainerStarted","Data":"35eaaebdd24c6b6198bc606e22e96ecc875e2509ad2f2cbd73c564331baddd0c"} Sep 08 07:04:47.807855 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.807823 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal" event={"ID":"6b5baafa68872d0e1a89c3b7fe397e3a","Type":"ContainerStarted","Data":"cde98212493381c3372e54d30a0492b72f429223d2a510661dbfd85d1d790640"} Sep 08 07:04:47.884654 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884615 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ace1cd1-322a-41ec-82df-53bc6914bd50-tmp-dir\") pod \"node-resolver-988kt\" (UID: \"4ace1cd1-322a-41ec-82df-53bc6914bd50\") " pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:47.884853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884670 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-run-ovn-kubernetes\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.884853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884700 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-cni-bin\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.884853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884732 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-kubelet-config\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:47.884853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884757 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/724e0b43-1d19-449a-8dcb-a3908b474a9f-env-overrides\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.884853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884783 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-run-systemd\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.884853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884808 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-log-socket\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.884853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884803 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-cni-bin\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.884853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884780 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-run-ovn-kubernetes\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.884853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884845 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-kubelet-config\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:47.884853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884848 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-run-systemd\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884871 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kntkk\" (UniqueName: \"kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk\") pod \"network-check-target-zts7r\" (UID: \"433a278e-8a9d-44ad-972c-f36e39dd23d0\") " pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884888 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-log-socket\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884932 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-node-log\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884965 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b05cd8e7-67da-40fe-a01d-d766c7d73ef9-host\") pod \"node-ca-drmx7\" (UID: \"b05cd8e7-67da-40fe-a01d-d766c7d73ef9\") " pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.884989 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa3811e3-5018-432a-8a05-49fabb23b1a3-host-slash\") pod \"iptables-alerter-b4q94\" (UID: \"aa3811e3-5018-432a-8a05-49fabb23b1a3\") " pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885020 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/724e0b43-1d19-449a-8dcb-a3908b474a9f-ovn-node-metrics-cert\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885043 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ace1cd1-322a-41ec-82df-53bc6914bd50-tmp-dir\") pod \"node-resolver-988kt\" (UID: \"4ace1cd1-322a-41ec-82df-53bc6914bd50\") " pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885093 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b05cd8e7-67da-40fe-a01d-d766c7d73ef9-host\") pod \"node-ca-drmx7\" (UID: \"b05cd8e7-67da-40fe-a01d-d766c7d73ef9\") " pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885094 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-node-log\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885044 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xws2p\" (UniqueName: \"kubernetes.io/projected/724e0b43-1d19-449a-8dcb-a3908b474a9f-kube-api-access-xws2p\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885146 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-dbus\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885178 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkthl\" (UniqueName: \"kubernetes.io/projected/aa3811e3-5018-432a-8a05-49fabb23b1a3-kube-api-access-zkthl\") pod \"iptables-alerter-b4q94\" (UID: \"aa3811e3-5018-432a-8a05-49fabb23b1a3\") " pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885205 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-var-lib-openvswitch\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885231 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885281 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xzxn\" (UniqueName: \"kubernetes.io/projected/b05cd8e7-67da-40fe-a01d-d766c7d73ef9-kube-api-access-9xzxn\") pod \"node-ca-drmx7\" (UID: \"b05cd8e7-67da-40fe-a01d-d766c7d73ef9\") " pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885307 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-kubelet\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.885396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885315 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/724e0b43-1d19-449a-8dcb-a3908b474a9f-env-overrides\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885337 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-slash\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885387 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-var-lib-openvswitch\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885410 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkx5z\" (UniqueName: \"kubernetes.io/projected/4ace1cd1-322a-41ec-82df-53bc6914bd50-kube-api-access-tkx5z\") pod \"node-resolver-988kt\" (UID: \"4ace1cd1-322a-41ec-82df-53bc6914bd50\") " pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885436 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-dbus\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885443 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/724e0b43-1d19-449a-8dcb-a3908b474a9f-ovnkube-script-lib\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885457 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa3811e3-5018-432a-8a05-49fabb23b1a3-host-slash\") pod \"iptables-alerter-b4q94\" (UID: \"aa3811e3-5018-432a-8a05-49fabb23b1a3\") " pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885522 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-slash\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885571 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-kubelet\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.885581 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885608 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b05cd8e7-67da-40fe-a01d-d766c7d73ef9-serviceca\") pod \"node-ca-drmx7\" (UID: \"b05cd8e7-67da-40fe-a01d-d766c7d73ef9\") " pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.885638 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret podName:fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:48.385618804 +0000 UTC m=+3.063581919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret") pod "global-pull-secret-syncer-gj26q" (UID: "fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43") : object "kube-system"/"original-pull-secret" not registered Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885674 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885709 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ace1cd1-322a-41ec-82df-53bc6914bd50-hosts-file\") pod \"node-resolver-988kt\" (UID: \"4ace1cd1-322a-41ec-82df-53bc6914bd50\") " pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885768 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ace1cd1-322a-41ec-82df-53bc6914bd50-hosts-file\") pod \"node-resolver-988kt\" (UID: \"4ace1cd1-322a-41ec-82df-53bc6914bd50\") " pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885765 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885816 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aa3811e3-5018-432a-8a05-49fabb23b1a3-iptables-alerter-script\") pod \"iptables-alerter-b4q94\" (UID: \"aa3811e3-5018-432a-8a05-49fabb23b1a3\") " pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:47.886232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885848 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-run-netns\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885874 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/724e0b43-1d19-449a-8dcb-a3908b474a9f-ovnkube-config\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885901 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-systemd-units\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885926 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-etc-openvswitch\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885932 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-run-netns\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885952 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-run-openvswitch\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885979 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-systemd-units\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.885982 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-run-ovn\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.886024 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-run-ovn\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.886025 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-cni-netd\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.886058 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-host-cni-netd\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.886097 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-etc-openvswitch\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.886101 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724e0b43-1d19-449a-8dcb-a3908b474a9f-run-openvswitch\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.886114 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b05cd8e7-67da-40fe-a01d-d766c7d73ef9-serviceca\") pod \"node-ca-drmx7\" (UID: \"b05cd8e7-67da-40fe-a01d-d766c7d73ef9\") " pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.886160 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/724e0b43-1d19-449a-8dcb-a3908b474a9f-ovnkube-script-lib\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.886414 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aa3811e3-5018-432a-8a05-49fabb23b1a3-iptables-alerter-script\") pod \"iptables-alerter-b4q94\" (UID: \"aa3811e3-5018-432a-8a05-49fabb23b1a3\") " pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:47.887002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.886477 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/724e0b43-1d19-449a-8dcb-a3908b474a9f-ovnkube-config\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.888050 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.888026 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/724e0b43-1d19-449a-8dcb-a3908b474a9f-ovn-node-metrics-cert\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.899048 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.899015 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 07:04:47.899048 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.899048 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 07:04:47.899198 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.899064 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kntkk for pod openshift-network-diagnostics/network-check-target-zts7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:04:47.899198 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:47.899139 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk podName:433a278e-8a9d-44ad-972c-f36e39dd23d0 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:48.399121308 +0000 UTC m=+3.077084421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kntkk" (UniqueName: "kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk") pod "network-check-target-zts7r" (UID: "433a278e-8a9d-44ad-972c-f36e39dd23d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:04:47.902095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.901812 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xzxn\" (UniqueName: \"kubernetes.io/projected/b05cd8e7-67da-40fe-a01d-d766c7d73ef9-kube-api-access-9xzxn\") pod \"node-ca-drmx7\" (UID: \"b05cd8e7-67da-40fe-a01d-d766c7d73ef9\") " pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:47.902095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.902043 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xws2p\" (UniqueName: \"kubernetes.io/projected/724e0b43-1d19-449a-8dcb-a3908b474a9f-kube-api-access-xws2p\") pod \"ovnkube-node-wsrt2\" (UID: \"724e0b43-1d19-449a-8dcb-a3908b474a9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:47.902556 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.902537 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkthl\" (UniqueName: \"kubernetes.io/projected/aa3811e3-5018-432a-8a05-49fabb23b1a3-kube-api-access-zkthl\") pod \"iptables-alerter-b4q94\" (UID: \"aa3811e3-5018-432a-8a05-49fabb23b1a3\") " pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:47.902860 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.902838 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkx5z\" (UniqueName: \"kubernetes.io/projected/4ace1cd1-322a-41ec-82df-53bc6914bd50-kube-api-access-tkx5z\") pod \"node-resolver-988kt\" (UID: \"4ace1cd1-322a-41ec-82df-53bc6914bd50\") " pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:47.979432 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.979382 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" Sep 08 07:04:47.988999 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:47.988977 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:04:48.000674 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.000640 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" Sep 08 07:04:48.007837 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.007811 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-678bx" Sep 08 07:04:48.017481 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.017461 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" Sep 08 07:04:48.024052 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.024036 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-988kt" Sep 08 07:04:48.031651 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.031620 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-drmx7" Sep 08 07:04:48.043467 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.043400 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b4q94" Sep 08 07:04:48.050173 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.050150 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:04:48.289870 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.289839 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:48.290059 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:48.290007 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:04:48.290115 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:48.290077 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs podName:b282970a-79b3-42c5-8200-192bf1b0ac43 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:49.290056651 +0000 UTC m=+3.968019773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs") pod "network-metrics-daemon-8k4kq" (UID: "b282970a-79b3-42c5-8200-192bf1b0ac43") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:04:48.391731 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.391616 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:48.391874 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:48.391778 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 07:04:48.391925 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:48.391873 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret podName:fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:49.391851811 +0000 UTC m=+4.069814924 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret") pod "global-pull-secret-syncer-gj26q" (UID: "fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43") : object "kube-system"/"original-pull-secret" not registered Sep 08 07:04:48.492618 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.492583 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kntkk\" (UniqueName: \"kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk\") pod \"network-check-target-zts7r\" (UID: \"433a278e-8a9d-44ad-972c-f36e39dd23d0\") " pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:48.492818 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:48.492782 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 07:04:48.492818 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:48.492810 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 07:04:48.492911 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:48.492823 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kntkk for pod openshift-network-diagnostics/network-check-target-zts7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:04:48.492911 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:48.492877 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk podName:433a278e-8a9d-44ad-972c-f36e39dd23d0 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:49.492861738 +0000 UTC m=+4.170824834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kntkk" (UniqueName: "kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk") pod "network-check-target-zts7r" (UID: "433a278e-8a9d-44ad-972c-f36e39dd23d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:04:48.715065 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.714999 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 06:59:48 +0000 UTC" deadline="2028-02-17 03:55:48.611645469 +0000 UTC" Sep 08 07:04:48.715065 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.715054 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12644h50m59.896595532s" Sep 08 07:04:48.803608 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.803434 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:48.803744 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:48.803698 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:04:48.806292 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:48.806263 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05cd8e7_67da_40fe_a01d_d766c7d73ef9.slice/crio-0ac141d0f86c51fc43d51def716b6871d1c60dbbfdd18852b0a324f435ab6679 WatchSource:0}: Error finding container 0ac141d0f86c51fc43d51def716b6871d1c60dbbfdd18852b0a324f435ab6679: Status 404 returned error can't find the container with id 0ac141d0f86c51fc43d51def716b6871d1c60dbbfdd18852b0a324f435ab6679 Sep 08 07:04:48.807825 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:48.807802 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc7868f_cf26_41b9_b221_a5d5563e6cc2.slice/crio-3cd4844222bb0ee59e92026982bb331fdb1ec5a49ab2210b4d6a479672893665 WatchSource:0}: Error finding container 3cd4844222bb0ee59e92026982bb331fdb1ec5a49ab2210b4d6a479672893665: Status 404 returned error can't find the container with id 3cd4844222bb0ee59e92026982bb331fdb1ec5a49ab2210b4d6a479672893665 Sep 08 07:04:48.810955 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:48.810919 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-drmx7" event={"ID":"b05cd8e7-67da-40fe-a01d-d766c7d73ef9","Type":"ContainerStarted","Data":"0ac141d0f86c51fc43d51def716b6871d1c60dbbfdd18852b0a324f435ab6679"} Sep 08 07:04:48.812036 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:48.812005 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d46ded_8a5c_46ba_abda_1359a34cd12c.slice/crio-fede39f493a01f74f010aa61539bf28415047994383d267f9b584a7487742316 WatchSource:0}: Error finding container fede39f493a01f74f010aa61539bf28415047994383d267f9b584a7487742316: Status 404 returned error can't find the container with id fede39f493a01f74f010aa61539bf28415047994383d267f9b584a7487742316 Sep 08 07:04:48.812885 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:48.812864 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa3811e3_5018_432a_8a05_49fabb23b1a3.slice/crio-14351ca16a545e29b505a360682d3d1b0d394faca078f5112373a5657ac269e7 WatchSource:0}: Error finding container 14351ca16a545e29b505a360682d3d1b0d394faca078f5112373a5657ac269e7: Status 404 returned error can't find the container with id 14351ca16a545e29b505a360682d3d1b0d394faca078f5112373a5657ac269e7 Sep 08 07:04:48.814415 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:48.814372 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod724e0b43_1d19_449a_8dcb_a3908b474a9f.slice/crio-50a1a70e69d3d87dde1aca9c48a5152d0f39d1bb06d5021cf2480af6fe4b3bbb WatchSource:0}: Error finding container 50a1a70e69d3d87dde1aca9c48a5152d0f39d1bb06d5021cf2480af6fe4b3bbb: Status 404 returned error can't find the container with id 50a1a70e69d3d87dde1aca9c48a5152d0f39d1bb06d5021cf2480af6fe4b3bbb Sep 08 07:04:48.816138 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:48.815704 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3cb711_52d1_4457_a3e2_ce1efbefbfe1.slice/crio-abcd267179fdbc2ee087f42820fc0fb0d979a4f33481e06ee9686e3ad2ed238a WatchSource:0}: Error finding container abcd267179fdbc2ee087f42820fc0fb0d979a4f33481e06ee9686e3ad2ed238a: Status 404 returned error can't find the container with id abcd267179fdbc2ee087f42820fc0fb0d979a4f33481e06ee9686e3ad2ed238a Sep 08 07:04:48.817710 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:48.817375 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ace1cd1_322a_41ec_82df_53bc6914bd50.slice/crio-4a97f849862d568d1dbe6cb634d35ee4d38dafc1d65ab248f4d17555567a061c WatchSource:0}: Error finding container 4a97f849862d568d1dbe6cb634d35ee4d38dafc1d65ab248f4d17555567a061c: Status 404 returned error can't find the container with id 4a97f849862d568d1dbe6cb634d35ee4d38dafc1d65ab248f4d17555567a061c Sep 08 07:04:48.819096 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:04:48.819074 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c085d6_4085_42a9_bf4c_ad13ad01f7e6.slice/crio-cacb59771b0b88f4875aa74fa2de5df53b474a8443be81be739dcf4ddbd7f0bd WatchSource:0}: Error finding container cacb59771b0b88f4875aa74fa2de5df53b474a8443be81be739dcf4ddbd7f0bd: Status 404 returned error can't find the container with id cacb59771b0b88f4875aa74fa2de5df53b474a8443be81be739dcf4ddbd7f0bd Sep 08 07:04:49.184244 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.184208 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 07:04:49.298077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.297560 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:49.298077 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:49.297734 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:04:49.298077 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:49.297806 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs podName:b282970a-79b3-42c5-8200-192bf1b0ac43 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:51.297784799 +0000 UTC m=+5.975747909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs") pod "network-metrics-daemon-8k4kq" (UID: "b282970a-79b3-42c5-8200-192bf1b0ac43") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:04:49.399245 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.398514 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:49.399245 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:49.398738 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 07:04:49.399245 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:49.398808 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret podName:fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:51.398788574 +0000 UTC m=+6.076751672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret") pod "global-pull-secret-syncer-gj26q" (UID: "fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43") : object "kube-system"/"original-pull-secret" not registered Sep 08 07:04:49.499915 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.499871 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kntkk\" (UniqueName: \"kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk\") pod \"network-check-target-zts7r\" (UID: \"433a278e-8a9d-44ad-972c-f36e39dd23d0\") " pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:49.500117 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:49.500049 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 07:04:49.500117 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:49.500077 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 07:04:49.500117 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:49.500092 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kntkk for pod openshift-network-diagnostics/network-check-target-zts7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:04:49.500322 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:49.500159 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk podName:433a278e-8a9d-44ad-972c-f36e39dd23d0 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:51.500139125 +0000 UTC m=+6.178102227 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kntkk" (UniqueName: "kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk") pod "network-check-target-zts7r" (UID: "433a278e-8a9d-44ad-972c-f36e39dd23d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:04:49.715497 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.715448 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 06:59:48 +0000 UTC" deadline="2028-06-18 15:32:22.53739581 +0000 UTC" Sep 08 07:04:49.715497 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.715493 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15584h27m32.821907263s" Sep 08 07:04:49.803819 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.803077 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:49.803819 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:49.803222 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:04:49.803819 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.803662 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:49.803819 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:49.803756 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:04:49.818598 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.818564 2571 generic.go:356] "Generic (PLEG): container finished" podID="e13a78121acfe93071c1c9eb406a02ec" containerID="7e2e66775a801ce45129ae19e04e0e6f9d17aa35673f0d56ba8622764a2b9e72" exitCode=0 Sep 08 07:04:49.818780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.818657 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" event={"ID":"e13a78121acfe93071c1c9eb406a02ec","Type":"ContainerDied","Data":"7e2e66775a801ce45129ae19e04e0e6f9d17aa35673f0d56ba8622764a2b9e72"} Sep 08 07:04:49.833546 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.833514 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-988kt" event={"ID":"4ace1cd1-322a-41ec-82df-53bc6914bd50","Type":"ContainerStarted","Data":"4a97f849862d568d1dbe6cb634d35ee4d38dafc1d65ab248f4d17555567a061c"} Sep 08 07:04:49.856752 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.856717 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ptdrw" event={"ID":"0d3cb711-52d1-4457-a3e2-ce1efbefbfe1","Type":"ContainerStarted","Data":"abcd267179fdbc2ee087f42820fc0fb0d979a4f33481e06ee9686e3ad2ed238a"} Sep 08 07:04:49.868176 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.868058 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" event={"ID":"724e0b43-1d19-449a-8dcb-a3908b474a9f","Type":"ContainerStarted","Data":"50a1a70e69d3d87dde1aca9c48a5152d0f39d1bb06d5021cf2480af6fe4b3bbb"} Sep 08 07:04:49.886116 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.886011 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b4q94" event={"ID":"aa3811e3-5018-432a-8a05-49fabb23b1a3","Type":"ContainerStarted","Data":"14351ca16a545e29b505a360682d3d1b0d394faca078f5112373a5657ac269e7"} Sep 08 07:04:49.890436 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.890354 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" event={"ID":"2bc7868f-cf26-41b9-b221-a5d5563e6cc2","Type":"ContainerStarted","Data":"3cd4844222bb0ee59e92026982bb331fdb1ec5a49ab2210b4d6a479672893665"} Sep 08 07:04:49.896195 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.896161 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal" event={"ID":"6b5baafa68872d0e1a89c3b7fe397e3a","Type":"ContainerStarted","Data":"561d834ab16da8a0baa446314c56e09c8b43547f054b65f44b3f7aa902ba6974"} Sep 08 07:04:49.909507 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.909423 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-678bx" event={"ID":"18c085d6-4085-42a9-bf4c-ad13ad01f7e6","Type":"ContainerStarted","Data":"cacb59771b0b88f4875aa74fa2de5df53b474a8443be81be739dcf4ddbd7f0bd"} Sep 08 07:04:49.926775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.926737 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" event={"ID":"73d46ded-8a5c-46ba-abda-1359a34cd12c","Type":"ContainerStarted","Data":"fede39f493a01f74f010aa61539bf28415047994383d267f9b584a7487742316"} Sep 08 07:04:49.953229 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:49.953190 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" event={"ID":"ea14dea9-8914-47a0-9ee9-e6f3da515e8d","Type":"ContainerStarted","Data":"b9687e072365a9d359fa1508d543c81699799c4bd41e650b00b03162f2527bd9"} Sep 08 07:04:50.803316 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:50.803275 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:50.803960 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:50.803428 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:04:50.977189 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:50.977149 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" event={"ID":"e13a78121acfe93071c1c9eb406a02ec","Type":"ContainerStarted","Data":"7c89f657740fc3ffd1fb7065bc6b1f57598cf273d8229a63f25d860b04df405b"} Sep 08 07:04:50.994658 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:50.993939 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-34.ec2.internal" podStartSLOduration=3.993916929 podStartE2EDuration="3.993916929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:04:49.912562919 +0000 UTC m=+4.590526059" watchObservedRunningTime="2026-09-08 07:04:50.993916929 +0000 UTC m=+5.671880041" Sep 08 07:04:51.316652 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:51.315858 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:51.316652 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:51.316103 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:04:51.316652 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:51.316171 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs podName:b282970a-79b3-42c5-8200-192bf1b0ac43 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:55.316150761 +0000 UTC m=+9.994113860 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs") pod "network-metrics-daemon-8k4kq" (UID: "b282970a-79b3-42c5-8200-192bf1b0ac43") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:04:51.417142 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:51.417101 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:51.417410 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:51.417298 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 07:04:51.417410 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:51.417391 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret podName:fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:55.417368676 +0000 UTC m=+10.095331787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret") pod "global-pull-secret-syncer-gj26q" (UID: "fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43") : object "kube-system"/"original-pull-secret" not registered Sep 08 07:04:51.518214 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:51.518061 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kntkk\" (UniqueName: \"kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk\") pod \"network-check-target-zts7r\" (UID: \"433a278e-8a9d-44ad-972c-f36e39dd23d0\") " pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:51.518415 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:51.518283 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 07:04:51.518415 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:51.518313 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 07:04:51.518415 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:51.518327 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kntkk for pod openshift-network-diagnostics/network-check-target-zts7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:04:51.518415 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:51.518402 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk podName:433a278e-8a9d-44ad-972c-f36e39dd23d0 nodeName:}" failed. No retries permitted until 2026-09-08 07:04:55.518380628 +0000 UTC m=+10.196343731 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kntkk" (UniqueName: "kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk") pod "network-check-target-zts7r" (UID: "433a278e-8a9d-44ad-972c-f36e39dd23d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:04:51.803436 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:51.803373 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:51.803965 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:51.803385 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:51.803965 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:51.803526 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:04:51.803965 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:51.803599 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:04:52.803305 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:52.802706 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:52.803305 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:52.802858 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:04:53.805512 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:53.804385 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:53.805512 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:53.804526 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:04:53.805512 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:53.805048 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:53.805512 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:53.805160 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:04:54.802935 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:54.802898 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:54.803098 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:54.803028 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:04:55.353871 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:55.353783 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:55.354406 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:55.354064 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:04:55.354406 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:55.354137 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs podName:b282970a-79b3-42c5-8200-192bf1b0ac43 nodeName:}" failed. No retries permitted until 2026-09-08 07:05:03.354114703 +0000 UTC m=+18.032077815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs") pod "network-metrics-daemon-8k4kq" (UID: "b282970a-79b3-42c5-8200-192bf1b0ac43") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:04:55.454125 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:55.454088 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:55.454301 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:55.454227 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 07:04:55.454301 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:55.454294 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret podName:fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43 nodeName:}" failed. No retries permitted until 2026-09-08 07:05:03.454280176 +0000 UTC m=+18.132243270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret") pod "global-pull-secret-syncer-gj26q" (UID: "fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43") : object "kube-system"/"original-pull-secret" not registered Sep 08 07:04:55.554706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:55.554663 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kntkk\" (UniqueName: \"kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk\") pod \"network-check-target-zts7r\" (UID: \"433a278e-8a9d-44ad-972c-f36e39dd23d0\") " pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:55.554962 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:55.554921 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 07:04:55.554962 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:55.554955 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 07:04:55.555090 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:55.554969 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kntkk for pod openshift-network-diagnostics/network-check-target-zts7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:04:55.555090 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:55.555040 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk podName:433a278e-8a9d-44ad-972c-f36e39dd23d0 nodeName:}" failed. No retries permitted until 2026-09-08 07:05:03.555019083 +0000 UTC m=+18.232982193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kntkk" (UniqueName: "kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk") pod "network-check-target-zts7r" (UID: "433a278e-8a9d-44ad-972c-f36e39dd23d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:04:55.804307 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:55.804265 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:55.804506 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:55.804437 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:04:55.804907 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:55.804883 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:55.805040 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:55.804989 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:04:56.803537 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:56.803494 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:56.804019 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:56.803635 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:04:57.803535 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:57.803087 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:57.803535 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:57.803244 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:57.803535 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:57.803245 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:04:57.803535 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:57.803374 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:04:58.803661 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:58.803628 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:04:58.804100 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:58.803772 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:04:59.803884 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:59.803657 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:04:59.804347 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:04:59.803712 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:04:59.804347 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:59.804002 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:04:59.804347 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:04:59.804148 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:00.803032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:00.802998 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:00.803200 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:00.803137 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:01.803342 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:01.803304 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:01.803759 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:01.803351 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:01.803759 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:01.803447 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:01.803759 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:01.803575 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:02.803016 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:02.802984 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:02.803248 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:02.803100 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:03.416697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:03.416669 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:03.417139 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:03.416794 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:05:03.417139 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:03.416859 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs podName:b282970a-79b3-42c5-8200-192bf1b0ac43 nodeName:}" failed. No retries permitted until 2026-09-08 07:05:19.416841767 +0000 UTC m=+34.094804867 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs") pod "network-metrics-daemon-8k4kq" (UID: "b282970a-79b3-42c5-8200-192bf1b0ac43") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:05:03.517359 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:03.517325 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:03.517537 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:03.517459 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 07:05:03.517537 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:03.517519 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret podName:fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43 nodeName:}" failed. No retries permitted until 2026-09-08 07:05:19.51750279 +0000 UTC m=+34.195465888 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret") pod "global-pull-secret-syncer-gj26q" (UID: "fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43") : object "kube-system"/"original-pull-secret" not registered Sep 08 07:05:03.618682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:03.618637 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kntkk\" (UniqueName: \"kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk\") pod \"network-check-target-zts7r\" (UID: \"433a278e-8a9d-44ad-972c-f36e39dd23d0\") " pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:03.618853 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:03.618780 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 07:05:03.618853 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:03.618797 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 07:05:03.618853 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:03.618806 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kntkk for pod openshift-network-diagnostics/network-check-target-zts7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:05:03.618993 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:03.618872 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk podName:433a278e-8a9d-44ad-972c-f36e39dd23d0 nodeName:}" failed. No retries permitted until 2026-09-08 07:05:19.618856691 +0000 UTC m=+34.296819788 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kntkk" (UniqueName: "kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk") pod "network-check-target-zts7r" (UID: "433a278e-8a9d-44ad-972c-f36e39dd23d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:05:03.802641 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:03.802606 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:03.802806 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:03.802735 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:03.802806 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:03.802791 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:03.802934 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:03.802896 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:04.802816 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:04.802782 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:04.803293 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:04.802910 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:05.804298 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:05.804261 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:05.804699 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:05.804381 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:05.804699 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:05.804442 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:05.804699 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:05.804535 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:06.802875 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:06.802834 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:06.803060 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:06.802990 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:07.803696 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:07.803530 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:07.804444 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:07.803577 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:07.804444 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:07.803776 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:07.804444 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:07.803845 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:08.014444 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.014407 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-drmx7" event={"ID":"b05cd8e7-67da-40fe-a01d-d766c7d73ef9","Type":"ContainerStarted","Data":"56bb116b0fc9a884056c961f467c952393aafd9a0f4d934870924be089d4d5dd"} Sep 08 07:05:08.016039 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.016012 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-678bx" event={"ID":"18c085d6-4085-42a9-bf4c-ad13ad01f7e6","Type":"ContainerStarted","Data":"6ea665fdfe6c93f48c26ac0bb9cdf240cc7fdf6ad5f7e49006fe1572f13fffa7"} Sep 08 07:05:08.017955 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.017928 2571 generic.go:356] "Generic (PLEG): container finished" podID="73d46ded-8a5c-46ba-abda-1359a34cd12c" containerID="3fdcc9d27843516e2ad63354cb4537e60b77263dcba2d8e5a9e4a49d78f11ffe" exitCode=0 Sep 08 07:05:08.018055 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.018006 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" event={"ID":"73d46ded-8a5c-46ba-abda-1359a34cd12c","Type":"ContainerDied","Data":"3fdcc9d27843516e2ad63354cb4537e60b77263dcba2d8e5a9e4a49d78f11ffe"} Sep 08 07:05:08.019866 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.019843 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" event={"ID":"ea14dea9-8914-47a0-9ee9-e6f3da515e8d","Type":"ContainerStarted","Data":"17fffd570ac2eb89e300852d119a0f015d0a8d954d0876aa1175e2b88b357544"} Sep 08 07:05:08.021529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.021481 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-988kt" event={"ID":"4ace1cd1-322a-41ec-82df-53bc6914bd50","Type":"ContainerStarted","Data":"a49ebb8f777e3011aa8d18b541c5787172c2ee1ca3a2de23801a6eba231dcabd"} Sep 08 07:05:08.023133 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.023111 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ptdrw" event={"ID":"0d3cb711-52d1-4457-a3e2-ce1efbefbfe1","Type":"ContainerStarted","Data":"87d6553a0f411fe89c618bbf2defb1ef0d5e9e2c3b429e87eb7ecf8b7edf4236"} Sep 08 07:05:08.026720 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.026698 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" event={"ID":"724e0b43-1d19-449a-8dcb-a3908b474a9f","Type":"ContainerStarted","Data":"581f1eee5fee07a4f803d0fc58d6ecf328ad1c981d6be6134276994ac601a379"} Sep 08 07:05:08.026824 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.026762 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" event={"ID":"724e0b43-1d19-449a-8dcb-a3908b474a9f","Type":"ContainerStarted","Data":"c43539d4a57ad55ab6eb8a025c37c9365c942d32e57156bb2e886eeb74bb60aa"} Sep 08 07:05:08.026824 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.026810 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" event={"ID":"724e0b43-1d19-449a-8dcb-a3908b474a9f","Type":"ContainerStarted","Data":"7b4da60dd8427fb410156da5cca0bdb9078331f5d1b2b7ae6d78334e83535b25"} Sep 08 07:05:08.026911 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.026853 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" event={"ID":"724e0b43-1d19-449a-8dcb-a3908b474a9f","Type":"ContainerStarted","Data":"6af7d083f0b217cc5549885b3eacfb395aa9711452d4693822c31172c258aeaa"} Sep 08 07:05:08.026911 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.026881 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" event={"ID":"724e0b43-1d19-449a-8dcb-a3908b474a9f","Type":"ContainerStarted","Data":"5f2bbeccfebd4435e9f6a3f38dbee80ec6ca6dc45e8a8facbe6036d7f295f398"} Sep 08 07:05:08.026997 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.026913 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" event={"ID":"724e0b43-1d19-449a-8dcb-a3908b474a9f","Type":"ContainerStarted","Data":"35649872ea1a909133dadfa218d2b8de89fafeac6d96867cb72ede5127725c62"} Sep 08 07:05:08.028344 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.028319 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" event={"ID":"2bc7868f-cf26-41b9-b221-a5d5563e6cc2","Type":"ContainerStarted","Data":"aad86de0ac1e35fff3be7116edd5de4df102302ab2b75c9fc23ffc37193fbc72"} Sep 08 07:05:08.029230 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.029180 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-drmx7" podStartSLOduration=3.711644572 podStartE2EDuration="22.029169674s" totalImagesPullingTime="18.317525102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:08.028954903 +0000 UTC m=+22.706918032" watchObservedRunningTime="2026-09-08 07:05:08.029169674 +0000 UTC m=+22.707132790" Sep 08 07:05:08.029335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.029292 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-34.ec2.internal" podStartSLOduration=21.029281225 podStartE2EDuration="21.029281225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:04:50.993746972 +0000 UTC m=+5.671710120" watchObservedRunningTime="2026-09-08 07:05:08.029281225 +0000 UTC m=+22.707244341" Sep 08 07:05:08.043580 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.043536 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ptdrw" podStartSLOduration=11.18159798 podStartE2EDuration="22.043518111s" totalImagesPullingTime="10.861920131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:08.043041692 +0000 UTC m=+22.721004828" watchObservedRunningTime="2026-09-08 07:05:08.043518111 +0000 UTC m=+22.721481228" Sep 08 07:05:08.061038 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.060993 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-678bx" podStartSLOduration=3.6381011340000002 podStartE2EDuration="22.060976784s" totalImagesPullingTime="18.42287565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:08.060737913 +0000 UTC m=+22.738701045" watchObservedRunningTime="2026-09-08 07:05:08.060976784 +0000 UTC m=+22.738939903" Sep 08 07:05:08.100895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.100851 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-988kt" podStartSLOduration=3.766436004 podStartE2EDuration="22.100835843s" totalImagesPullingTime="18.334399839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:08.100322807 +0000 UTC m=+22.778285948" watchObservedRunningTime="2026-09-08 07:05:08.100835843 +0000 UTC m=+22.778798958" Sep 08 07:05:08.117474 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.117428 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hlpl7" podStartSLOduration=3.773046972 podStartE2EDuration="22.117417018s" totalImagesPullingTime="18.344370046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:08.117173207 +0000 UTC m=+22.795136342" watchObservedRunningTime="2026-09-08 07:05:08.117417018 +0000 UTC m=+22.795380134" Sep 08 07:05:08.770054 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.769859 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 08 07:05:08.802899 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:08.802873 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:08.803009 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:08.802987 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:09.033027 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:09.032990 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b4q94" event={"ID":"aa3811e3-5018-432a-8a05-49fabb23b1a3","Type":"ContainerStarted","Data":"5a417119f5e0cd0e1d72b14531c77fd34ab7e621ad80159b70962cbc0205e6d6"} Sep 08 07:05:09.035164 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:09.035131 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" event={"ID":"2bc7868f-cf26-41b9-b221-a5d5563e6cc2","Type":"ContainerStarted","Data":"542d11a02c9b5ef64fd0c89529b0eef5bb9ffde9b37d8d931909d5839e3b92cd"} Sep 08 07:05:09.048532 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:09.048489 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b4q94" podStartSLOduration=4.70895873 podStartE2EDuration="23.048474243s" totalImagesPullingTime="18.339515513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:09.048267003 +0000 UTC m=+23.726230148" watchObservedRunningTime="2026-09-08 07:05:09.048474243 +0000 UTC m=+23.726437358" Sep 08 07:05:09.737688 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:09.737570 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T07:05:08.770041851Z","UUID":"b66490e6-6db8-4b41-b54d-275c9cdabede","Handler":null,"Name":"","Endpoint":""} Sep 08 07:05:09.739855 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:09.739827 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 08 07:05:09.740001 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:09.739872 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 08 07:05:09.803099 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:09.803068 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:09.803291 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:09.803068 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:09.803291 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:09.803191 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:09.803291 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:09.803246 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:10.040286 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:10.040184 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" event={"ID":"2bc7868f-cf26-41b9-b221-a5d5563e6cc2","Type":"ContainerStarted","Data":"c96def44289454d2560c7d77d7e5c0b607a6ff50801d4f22d941ba140ddbdbc1"} Sep 08 07:05:10.046243 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:10.045787 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" event={"ID":"724e0b43-1d19-449a-8dcb-a3908b474a9f","Type":"ContainerStarted","Data":"0989e3a3acd8105fbc39ac9bf63d8b5c93e4f0976bbcc1559976e88fea7005f8"} Sep 08 07:05:10.057639 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:10.057590 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4twh" podStartSLOduration=3.441523775 podStartE2EDuration="24.057572411s" totalImagesPullingTime="20.616048636s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:10.057509271 +0000 UTC m=+24.735472410" watchObservedRunningTime="2026-09-08 07:05:10.057572411 +0000 UTC m=+24.735535530" Sep 08 07:05:10.803010 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:10.802977 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:10.803196 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:10.803101 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:11.803604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:11.803568 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:11.804094 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:11.803568 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:11.804094 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:11.803699 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:11.804094 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:11.803761 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:12.046389 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:12.046360 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:05:12.047046 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:12.047023 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:05:12.803799 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:12.803587 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:12.804240 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:12.803842 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:13.054520 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:13.054416 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" event={"ID":"724e0b43-1d19-449a-8dcb-a3908b474a9f","Type":"ContainerStarted","Data":"77d8c2c853010b27d1c418df85837a3f9f890555c7e58faeaadc0238cc519d13"} Sep 08 07:05:13.054711 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:13.054694 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:05:13.054795 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:13.054777 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:05:13.056264 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:13.056241 2571 generic.go:356] "Generic (PLEG): container finished" podID="73d46ded-8a5c-46ba-abda-1359a34cd12c" containerID="b9408f6cdc32737e1d6ae60415b98d8f51dfd3abfba70c5d5a1b00cb39fa10c1" exitCode=0 Sep 08 07:05:13.056369 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:13.056283 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" event={"ID":"73d46ded-8a5c-46ba-abda-1359a34cd12c","Type":"ContainerDied","Data":"b9408f6cdc32737e1d6ae60415b98d8f51dfd3abfba70c5d5a1b00cb39fa10c1"} Sep 08 07:05:13.070720 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:13.070702 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:05:13.080065 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:13.080030 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" podStartSLOduration=8.736228807 podStartE2EDuration="27.080017948s" totalImagesPullingTime="18.343789141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:13.078255578 +0000 UTC m=+27.756218705" watchObservedRunningTime="2026-09-08 07:05:13.080017948 +0000 UTC m=+27.757981065" Sep 08 07:05:13.802678 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:13.802650 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:13.802845 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:13.802650 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:13.802845 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:13.802777 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:13.803048 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:13.802853 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:14.061275 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:14.061178 2571 generic.go:356] "Generic (PLEG): container finished" podID="73d46ded-8a5c-46ba-abda-1359a34cd12c" containerID="b153824630035b8c5199f4aa35208b74e9a1069e255631720e52d819cb023b77" exitCode=0 Sep 08 07:05:14.061275 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:14.061254 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" event={"ID":"73d46ded-8a5c-46ba-abda-1359a34cd12c","Type":"ContainerDied","Data":"b153824630035b8c5199f4aa35208b74e9a1069e255631720e52d819cb023b77"} Sep 08 07:05:14.061890 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:14.061813 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:05:14.076555 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:14.076531 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:05:14.803149 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:14.803126 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:14.803263 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:14.803245 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:15.066271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:15.066178 2571 generic.go:356] "Generic (PLEG): container finished" podID="73d46ded-8a5c-46ba-abda-1359a34cd12c" containerID="3b59133ec61178cc0737f1f2df597c608ffadeacb205f60200d7d1e9e5c7c26c" exitCode=0 Sep 08 07:05:15.066271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:15.066261 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" event={"ID":"73d46ded-8a5c-46ba-abda-1359a34cd12c","Type":"ContainerDied","Data":"3b59133ec61178cc0737f1f2df597c608ffadeacb205f60200d7d1e9e5c7c26c"} Sep 08 07:05:15.804250 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:15.804201 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:15.804451 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:15.804329 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:15.804451 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:15.804386 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:15.804569 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:15.804475 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:16.803602 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:16.803570 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:16.804157 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:16.803690 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:17.621674 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:17.621640 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:05:17.621850 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:17.621835 2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 08 07:05:17.622283 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:17.622254 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ptdrw" Sep 08 07:05:17.803172 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:17.803139 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:17.803373 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:17.803296 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:17.803373 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:17.803348 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:17.803494 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:17.803477 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:18.803282 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:18.803249 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:18.803695 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:18.803371 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:19.430223 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:19.430190 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:19.430325 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:19.430311 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:05:19.430374 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:19.430364 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs podName:b282970a-79b3-42c5-8200-192bf1b0ac43 nodeName:}" failed. No retries permitted until 2026-09-08 07:05:51.430351728 +0000 UTC m=+66.108314822 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs") pod "network-metrics-daemon-8k4kq" (UID: "b282970a-79b3-42c5-8200-192bf1b0ac43") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 07:05:19.531317 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:19.531283 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:19.531457 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:19.531428 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 07:05:19.531502 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:19.531488 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret podName:fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43 nodeName:}" failed. No retries permitted until 2026-09-08 07:05:51.531474647 +0000 UTC m=+66.209437754 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret") pod "global-pull-secret-syncer-gj26q" (UID: "fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43") : object "kube-system"/"original-pull-secret" not registered Sep 08 07:05:19.632475 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:19.632448 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kntkk\" (UniqueName: \"kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk\") pod \"network-check-target-zts7r\" (UID: \"433a278e-8a9d-44ad-972c-f36e39dd23d0\") " pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:19.632587 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:19.632570 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 07:05:19.632587 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:19.632587 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 07:05:19.632663 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:19.632596 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kntkk for pod openshift-network-diagnostics/network-check-target-zts7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:05:19.632663 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:19.632642 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk podName:433a278e-8a9d-44ad-972c-f36e39dd23d0 nodeName:}" failed. No retries permitted until 2026-09-08 07:05:51.632630023 +0000 UTC m=+66.310593117 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kntkk" (UniqueName: "kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk") pod "network-check-target-zts7r" (UID: "433a278e-8a9d-44ad-972c-f36e39dd23d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 07:05:19.803393 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:19.803355 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:19.803831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:19.803369 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:19.803831 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:19.803489 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:19.803831 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:19.803530 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:20.080415 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:20.080336 2571 generic.go:356] "Generic (PLEG): container finished" podID="73d46ded-8a5c-46ba-abda-1359a34cd12c" containerID="58f457ee3f12781698d6af0ca8517cd2c59808aa94eaebd7980d510c53a1115e" exitCode=0 Sep 08 07:05:20.080415 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:20.080389 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" event={"ID":"73d46ded-8a5c-46ba-abda-1359a34cd12c","Type":"ContainerDied","Data":"58f457ee3f12781698d6af0ca8517cd2c59808aa94eaebd7980d510c53a1115e"} Sep 08 07:05:20.803054 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:20.803017 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:20.803241 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:20.803128 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:21.086996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:21.086682 2571 generic.go:356] "Generic (PLEG): container finished" podID="73d46ded-8a5c-46ba-abda-1359a34cd12c" containerID="1cf5546454a3ec901b77b951d0f92373587d0bc7c277a56f7cfd8077799b17e8" exitCode=0 Sep 08 07:05:21.087433 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:21.086746 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" event={"ID":"73d46ded-8a5c-46ba-abda-1359a34cd12c","Type":"ContainerDied","Data":"1cf5546454a3ec901b77b951d0f92373587d0bc7c277a56f7cfd8077799b17e8"} Sep 08 07:05:21.374412 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:21.374320 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gj26q"] Sep 08 07:05:21.374573 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:21.374452 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:21.374573 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:21.374544 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:21.376851 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:21.376820 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zts7r"] Sep 08 07:05:21.377012 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:21.376957 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:21.377087 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:21.377066 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:21.377847 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:21.377805 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8k4kq"] Sep 08 07:05:21.377963 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:21.377914 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:21.378061 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:21.378024 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:22.093409 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:22.093378 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" event={"ID":"73d46ded-8a5c-46ba-abda-1359a34cd12c","Type":"ContainerStarted","Data":"b52def48d51ff5eaf212bedbb8ce10f5c6722b7fc52d734e9a86f133901a5996"} Sep 08 07:05:22.116243 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:22.116180 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dtfw9" podStartSLOduration=7.637840087 podStartE2EDuration="36.116162684s" totalImagesPullingTime="28.478322597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:22.114258284 +0000 UTC m=+36.792221418" watchObservedRunningTime="2026-09-08 07:05:22.116162684 +0000 UTC m=+36.794125800" Sep 08 07:05:22.802940 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:22.802900 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:22.802940 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:22.802946 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:22.803161 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:22.803040 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:22.803161 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:22.803129 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:23.803052 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:23.802819 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:23.803426 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:23.803091 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8k4kq" podUID="b282970a-79b3-42c5-8200-192bf1b0ac43" Sep 08 07:05:24.802853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:24.802818 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:24.803029 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:24.802819 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:24.803029 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:24.802986 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gj26q" podUID="fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43" Sep 08 07:05:24.803347 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:24.803066 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zts7r" podUID="433a278e-8a9d-44ad-972c-f36e39dd23d0" Sep 08 07:05:25.056557 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.056532 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-34.ec2.internal" event="NodeReady" Sep 08 07:05:25.056765 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.056677 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 08 07:05:25.090661 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.090626 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bf64cffd6-qdfp2"] Sep 08 07:05:25.109280 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.109245 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l246k"] Sep 08 07:05:25.109435 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.109352 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.111748 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.111720 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cpwg8\"" Sep 08 07:05:25.111926 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.111776 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 08 07:05:25.111926 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.111823 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 08 07:05:25.111926 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.111934 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 08 07:05:25.120111 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.120046 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 08 07:05:25.126169 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.126148 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rtxzw"] Sep 08 07:05:25.126299 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.126284 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.128535 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.128517 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 08 07:05:25.128785 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.128770 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmqzm\"" Sep 08 07:05:25.128874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.128802 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 08 07:05:25.150697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.150674 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bf64cffd6-qdfp2"] Sep 08 07:05:25.150697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.150704 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l246k"] Sep 08 07:05:25.150855 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.150723 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rtxzw"] Sep 08 07:05:25.150855 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.150738 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9nwvl"] Sep 08 07:05:25.150855 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.150786 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rtxzw" Sep 08 07:05:25.153324 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.153305 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 08 07:05:25.153324 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.153316 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qkwpw\"" Sep 08 07:05:25.153324 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.153305 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 08 07:05:25.153545 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.153338 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 08 07:05:25.177091 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.177065 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9nwvl"] Sep 08 07:05:25.177226 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.177168 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.180172 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.180148 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 08 07:05:25.180297 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.180192 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 08 07:05:25.180297 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.180153 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lv5z5\"" Sep 08 07:05:25.180297 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.180234 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 08 07:05:25.180297 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.180273 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 08 07:05:25.275753 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.275724 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-registry-certificates\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.275895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.275757 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-bound-sa-token\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.275895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.275782 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-trusted-ca\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.275895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.275837 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1925de9-7fbe-4cad-9181-eab4f6240e73-cert\") pod \"ingress-canary-rtxzw\" (UID: \"e1925de9-7fbe-4cad-9181-eab4f6240e73\") " pod="openshift-ingress-canary/ingress-canary-rtxzw" Sep 08 07:05:25.275895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.275867 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtf6g\" (UniqueName: \"kubernetes.io/projected/e1925de9-7fbe-4cad-9181-eab4f6240e73-kube-api-access-rtf6g\") pod \"ingress-canary-rtxzw\" (UID: \"e1925de9-7fbe-4cad-9181-eab4f6240e73\") " pod="openshift-ingress-canary/ingress-canary-rtxzw" Sep 08 07:05:25.275895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.275885 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/594b1074-cee3-42b8-9ec1-ded1a72984c4-config-volume\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.276084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.275904 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/594b1074-cee3-42b8-9ec1-ded1a72984c4-metrics-tls\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.276084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.275923 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whmmk\" (UniqueName: \"kubernetes.io/projected/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-kube-api-access-whmmk\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.276084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.275945 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6f012a0-c7b8-4f72-93f4-c337275d2046-data-volume\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.276084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.275964 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/594b1074-cee3-42b8-9ec1-ded1a72984c4-tmp-dir\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.276084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.276019 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnk5f\" (UniqueName: \"kubernetes.io/projected/594b1074-cee3-42b8-9ec1-ded1a72984c4-kube-api-access-hnk5f\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.276084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.276042 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-image-registry-private-configuration\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.276084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.276072 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-installation-pull-secrets\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.276366 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.276087 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hxlc\" (UniqueName: \"kubernetes.io/projected/c6f012a0-c7b8-4f72-93f4-c337275d2046-kube-api-access-4hxlc\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.276366 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.276118 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c6f012a0-c7b8-4f72-93f4-c337275d2046-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.276366 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.276161 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c6f012a0-c7b8-4f72-93f4-c337275d2046-crio-socket\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.276366 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.276178 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c6f012a0-c7b8-4f72-93f4-c337275d2046-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.276366 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.276204 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-registry-tls\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.276366 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.276246 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-ca-trust-extracted\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.377372 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377289 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1925de9-7fbe-4cad-9181-eab4f6240e73-cert\") pod \"ingress-canary-rtxzw\" (UID: \"e1925de9-7fbe-4cad-9181-eab4f6240e73\") " pod="openshift-ingress-canary/ingress-canary-rtxzw" Sep 08 07:05:25.377372 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377327 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtf6g\" (UniqueName: \"kubernetes.io/projected/e1925de9-7fbe-4cad-9181-eab4f6240e73-kube-api-access-rtf6g\") pod \"ingress-canary-rtxzw\" (UID: \"e1925de9-7fbe-4cad-9181-eab4f6240e73\") " pod="openshift-ingress-canary/ingress-canary-rtxzw" Sep 08 07:05:25.377372 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377344 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/594b1074-cee3-42b8-9ec1-ded1a72984c4-config-volume\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.377372 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377360 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/594b1074-cee3-42b8-9ec1-ded1a72984c4-metrics-tls\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.377682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377385 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whmmk\" (UniqueName: \"kubernetes.io/projected/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-kube-api-access-whmmk\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.377682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377417 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6f012a0-c7b8-4f72-93f4-c337275d2046-data-volume\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.377682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377439 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/594b1074-cee3-42b8-9ec1-ded1a72984c4-tmp-dir\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.377682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377482 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnk5f\" (UniqueName: \"kubernetes.io/projected/594b1074-cee3-42b8-9ec1-ded1a72984c4-kube-api-access-hnk5f\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.377682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377509 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-image-registry-private-configuration\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.377682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377553 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-installation-pull-secrets\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.377682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377584 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hxlc\" (UniqueName: \"kubernetes.io/projected/c6f012a0-c7b8-4f72-93f4-c337275d2046-kube-api-access-4hxlc\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.377682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377614 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c6f012a0-c7b8-4f72-93f4-c337275d2046-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.377682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377658 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c6f012a0-c7b8-4f72-93f4-c337275d2046-crio-socket\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.377682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377681 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c6f012a0-c7b8-4f72-93f4-c337275d2046-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.378104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377712 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-registry-tls\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.378104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377747 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-ca-trust-extracted\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.378104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377771 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-registry-certificates\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.378104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377796 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-bound-sa-token\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.378104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377811 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6f012a0-c7b8-4f72-93f4-c337275d2046-data-volume\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.378104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377829 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-trusted-ca\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.378104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377947 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c6f012a0-c7b8-4f72-93f4-c337275d2046-crio-socket\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.378104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.377973 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/594b1074-cee3-42b8-9ec1-ded1a72984c4-config-volume\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.378495 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.378239 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-ca-trust-extracted\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.379280 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.378753 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/594b1074-cee3-42b8-9ec1-ded1a72984c4-tmp-dir\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.379280 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.378790 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-trusted-ca\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.379280 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.378921 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-registry-certificates\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.380151 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.379589 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c6f012a0-c7b8-4f72-93f4-c337275d2046-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.382798 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.382773 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/594b1074-cee3-42b8-9ec1-ded1a72984c4-metrics-tls\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.382895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.382776 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c6f012a0-c7b8-4f72-93f4-c337275d2046-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.382895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.382816 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-installation-pull-secrets\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.382895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.382845 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-registry-tls\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.383063 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.382978 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1925de9-7fbe-4cad-9181-eab4f6240e73-cert\") pod \"ingress-canary-rtxzw\" (UID: \"e1925de9-7fbe-4cad-9181-eab4f6240e73\") " pod="openshift-ingress-canary/ingress-canary-rtxzw" Sep 08 07:05:25.383131 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.383113 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-image-registry-private-configuration\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.385736 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.385710 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtf6g\" (UniqueName: \"kubernetes.io/projected/e1925de9-7fbe-4cad-9181-eab4f6240e73-kube-api-access-rtf6g\") pod \"ingress-canary-rtxzw\" (UID: \"e1925de9-7fbe-4cad-9181-eab4f6240e73\") " pod="openshift-ingress-canary/ingress-canary-rtxzw" Sep 08 07:05:25.386001 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.385978 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnk5f\" (UniqueName: \"kubernetes.io/projected/594b1074-cee3-42b8-9ec1-ded1a72984c4-kube-api-access-hnk5f\") pod \"dns-default-l246k\" (UID: \"594b1074-cee3-42b8-9ec1-ded1a72984c4\") " pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.386374 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.386354 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whmmk\" (UniqueName: \"kubernetes.io/projected/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-kube-api-access-whmmk\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.387168 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.387134 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/733d6b54-2cf9-4d27-adb9-699a7e4a9e59-bound-sa-token\") pod \"image-registry-5bf64cffd6-qdfp2\" (UID: \"733d6b54-2cf9-4d27-adb9-699a7e4a9e59\") " pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.387168 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.387166 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hxlc\" (UniqueName: \"kubernetes.io/projected/c6f012a0-c7b8-4f72-93f4-c337275d2046-kube-api-access-4hxlc\") pod \"insights-runtime-extractor-9nwvl\" (UID: \"c6f012a0-c7b8-4f72-93f4-c337275d2046\") " pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.424183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.424151 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:25.434316 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.434293 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l246k" Sep 08 07:05:25.459301 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.459275 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rtxzw" Sep 08 07:05:25.485599 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.485562 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9nwvl" Sep 08 07:05:25.736004 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.735971 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bf64cffd6-qdfp2"] Sep 08 07:05:25.738629 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.738589 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l246k"] Sep 08 07:05:25.739658 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:25.739635 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733d6b54_2cf9_4d27_adb9_699a7e4a9e59.slice/crio-fd910bade4e0aa06c9dc6012d747304b6364ea5b46077247e4fb9b5e9ab6eadb WatchSource:0}: Error finding container fd910bade4e0aa06c9dc6012d747304b6364ea5b46077247e4fb9b5e9ab6eadb: Status 404 returned error can't find the container with id fd910bade4e0aa06c9dc6012d747304b6364ea5b46077247e4fb9b5e9ab6eadb Sep 08 07:05:25.740893 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:25.740870 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod594b1074_cee3_42b8_9ec1_ded1a72984c4.slice/crio-04984b9c48098c3b01fd25b7e458feb7279a01f66ca95d7e90c47dda73793b14 WatchSource:0}: Error finding container 04984b9c48098c3b01fd25b7e458feb7279a01f66ca95d7e90c47dda73793b14: Status 404 returned error can't find the container with id 04984b9c48098c3b01fd25b7e458feb7279a01f66ca95d7e90c47dda73793b14 Sep 08 07:05:25.776409 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.776382 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rtxzw"] Sep 08 07:05:25.781399 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:25.781378 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1925de9_7fbe_4cad_9181_eab4f6240e73.slice/crio-2bc9dfd346af9a7dc5d9001fd73469bde3f0100c399d98bc028bc5b9416576f3 WatchSource:0}: Error finding container 2bc9dfd346af9a7dc5d9001fd73469bde3f0100c399d98bc028bc5b9416576f3: Status 404 returned error can't find the container with id 2bc9dfd346af9a7dc5d9001fd73469bde3f0100c399d98bc028bc5b9416576f3 Sep 08 07:05:25.803893 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.803867 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:25.806627 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.806601 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 07:05:25.806921 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.806867 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ldq7w\"" Sep 08 07:05:25.807472 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:25.807454 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9nwvl"] Sep 08 07:05:25.810904 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:25.810874 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f012a0_c7b8_4f72_93f4_c337275d2046.slice/crio-ff1adc25492b685dd037c15496fef8613964d5f16b903cf51d4248a7bdfa3ffc WatchSource:0}: Error finding container ff1adc25492b685dd037c15496fef8613964d5f16b903cf51d4248a7bdfa3ffc: Status 404 returned error can't find the container with id ff1adc25492b685dd037c15496fef8613964d5f16b903cf51d4248a7bdfa3ffc Sep 08 07:05:26.105475 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.105401 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" event={"ID":"733d6b54-2cf9-4d27-adb9-699a7e4a9e59","Type":"ContainerStarted","Data":"56ea27219408a00dbac6a6fbbed04a82cfb2d08d5126cd6be195bbd08aaa91e5"} Sep 08 07:05:26.105475 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.105460 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" event={"ID":"733d6b54-2cf9-4d27-adb9-699a7e4a9e59","Type":"ContainerStarted","Data":"fd910bade4e0aa06c9dc6012d747304b6364ea5b46077247e4fb9b5e9ab6eadb"} Sep 08 07:05:26.105658 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.105522 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:26.106948 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.106925 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9nwvl" event={"ID":"c6f012a0-c7b8-4f72-93f4-c337275d2046","Type":"ContainerStarted","Data":"a2ffd47b85684a2d99bf5f96228d31cea9c239c54f44ef2fb51a29137a94eb42"} Sep 08 07:05:26.107040 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.106968 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9nwvl" event={"ID":"c6f012a0-c7b8-4f72-93f4-c337275d2046","Type":"ContainerStarted","Data":"ff1adc25492b685dd037c15496fef8613964d5f16b903cf51d4248a7bdfa3ffc"} Sep 08 07:05:26.108085 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.108061 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l246k" event={"ID":"594b1074-cee3-42b8-9ec1-ded1a72984c4","Type":"ContainerStarted","Data":"04984b9c48098c3b01fd25b7e458feb7279a01f66ca95d7e90c47dda73793b14"} Sep 08 07:05:26.109176 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.109152 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rtxzw" event={"ID":"e1925de9-7fbe-4cad-9181-eab4f6240e73","Type":"ContainerStarted","Data":"2bc9dfd346af9a7dc5d9001fd73469bde3f0100c399d98bc028bc5b9416576f3"} Sep 08 07:05:26.127100 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.127062 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" podStartSLOduration=6.127048455 podStartE2EDuration="6.127048455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:26.125759753 +0000 UTC m=+40.803722870" watchObservedRunningTime="2026-09-08 07:05:26.127048455 +0000 UTC m=+40.805011571" Sep 08 07:05:26.802646 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.802604 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:26.802837 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.802612 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:26.805331 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.805305 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 07:05:26.805749 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.805414 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 07:05:26.806438 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.806414 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 07:05:26.806563 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:26.806514 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7mcm7\"" Sep 08 07:05:27.114578 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:27.114539 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9nwvl" event={"ID":"c6f012a0-c7b8-4f72-93f4-c337275d2046","Type":"ContainerStarted","Data":"4c71202668e4062f7bc1c9fb7e00ae2a7a733d9e14bfecd513ebba8c2cb7dc8a"} Sep 08 07:05:30.126781 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:30.126682 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rtxzw" event={"ID":"e1925de9-7fbe-4cad-9181-eab4f6240e73","Type":"ContainerStarted","Data":"473d21cd9a6aebfbc8b9e8517b813dbc2ae7fc8915cdeab0817d7055ac2c4a98"} Sep 08 07:05:30.128840 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:30.128810 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9nwvl" event={"ID":"c6f012a0-c7b8-4f72-93f4-c337275d2046","Type":"ContainerStarted","Data":"e83ea9113e1217ae2ea7bd8f1921334588221acb15a24e3134da74af697aef07"} Sep 08 07:05:30.130576 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:30.130551 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l246k" event={"ID":"594b1074-cee3-42b8-9ec1-ded1a72984c4","Type":"ContainerStarted","Data":"ffb8406b5b3cbaf62d84bc2dfa08c583125101b7007c73160b4ca8130959cd29"} Sep 08 07:05:30.130694 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:30.130589 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l246k" event={"ID":"594b1074-cee3-42b8-9ec1-ded1a72984c4","Type":"ContainerStarted","Data":"6b624efd1dcd8bc60fb2dd6fe311ad4dcdcea6e65fcf9209450904d1e30192e3"} Sep 08 07:05:30.130694 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:30.130625 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l246k" Sep 08 07:05:30.141428 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:30.141378 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rtxzw" podStartSLOduration=1.582832152 podStartE2EDuration="5.141365181s" totalImagesPullingTime="3.558533029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:30.140686399 +0000 UTC m=+44.818649514" watchObservedRunningTime="2026-09-08 07:05:30.141365181 +0000 UTC m=+44.819328293" Sep 08 07:05:30.157242 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:30.157179 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9nwvl" podStartSLOduration=1.58624923 podStartE2EDuration="5.157161021s" totalImagesPullingTime="3.570911791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:05:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:30.156407749 +0000 UTC m=+44.834370911" watchObservedRunningTime="2026-09-08 07:05:30.157161021 +0000 UTC m=+44.835124139" Sep 08 07:05:30.172847 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:30.172789 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l246k" podStartSLOduration=1.573235336 podStartE2EDuration="5.172771002s" totalImagesPullingTime="3.599535666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:30.171863631 +0000 UTC m=+44.849826776" watchObservedRunningTime="2026-09-08 07:05:30.172771002 +0000 UTC m=+44.850734118" Sep 08 07:05:36.074515 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.074478 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f84dfd9d5-ms5zl"] Sep 08 07:05:36.106559 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.106505 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f84dfd9d5-ms5zl"] Sep 08 07:05:36.106721 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.106641 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.109775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.109754 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mxlnj\"" Sep 08 07:05:36.109894 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.109755 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 08 07:05:36.110037 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.110021 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 08 07:05:36.110753 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.110738 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 08 07:05:36.110810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.110771 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 08 07:05:36.110913 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.110891 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 08 07:05:36.111037 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.110996 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 08 07:05:36.111108 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.111028 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 08 07:05:36.114973 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.114955 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 08 07:05:36.153647 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.153615 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-serving-cert\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.153647 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.153646 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-oauth-config\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.153847 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.153667 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-config\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.153847 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.153686 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-trusted-ca-bundle\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.153847 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.153729 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlcjz\" (UniqueName: \"kubernetes.io/projected/ed578ab9-2f2f-4c21-abb4-6effbd18011d-kube-api-access-xlcjz\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.153847 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.153815 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-oauth-serving-cert\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.153847 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.153837 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-service-ca\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.255041 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.255004 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-oauth-serving-cert\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.255041 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.255042 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-service-ca\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.255262 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.255088 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-serving-cert\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.255262 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.255106 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-oauth-config\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.255262 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.255122 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-config\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.255262 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.255138 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-trusted-ca-bundle\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.255262 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.255154 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlcjz\" (UniqueName: \"kubernetes.io/projected/ed578ab9-2f2f-4c21-abb4-6effbd18011d-kube-api-access-xlcjz\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.255949 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.255919 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-oauth-serving-cert\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.256060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.255919 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-service-ca\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.256060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.256021 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-config\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.256206 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.256187 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-trusted-ca-bundle\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.258126 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.258105 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-serving-cert\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.258259 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.258236 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-oauth-config\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.263743 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.263719 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlcjz\" (UniqueName: \"kubernetes.io/projected/ed578ab9-2f2f-4c21-abb4-6effbd18011d-kube-api-access-xlcjz\") pod \"console-f84dfd9d5-ms5zl\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.416107 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.416015 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:36.701484 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:36.701448 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f84dfd9d5-ms5zl"] Sep 08 07:05:36.704801 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:36.704772 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded578ab9_2f2f_4c21_abb4_6effbd18011d.slice/crio-f357b10d393b01dfa4dc5173d3a103a683ad87bcaa27c2e3fd50cf6583b023ee WatchSource:0}: Error finding container f357b10d393b01dfa4dc5173d3a103a683ad87bcaa27c2e3fd50cf6583b023ee: Status 404 returned error can't find the container with id f357b10d393b01dfa4dc5173d3a103a683ad87bcaa27c2e3fd50cf6583b023ee Sep 08 07:05:37.153589 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:37.153499 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f84dfd9d5-ms5zl" event={"ID":"ed578ab9-2f2f-4c21-abb4-6effbd18011d","Type":"ContainerStarted","Data":"f357b10d393b01dfa4dc5173d3a103a683ad87bcaa27c2e3fd50cf6583b023ee"} Sep 08 07:05:40.135999 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.135967 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l246k" Sep 08 07:05:40.164946 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.164912 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f84dfd9d5-ms5zl" event={"ID":"ed578ab9-2f2f-4c21-abb4-6effbd18011d","Type":"ContainerStarted","Data":"1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55"} Sep 08 07:05:40.190605 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.190553 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f84dfd9d5-ms5zl" podStartSLOduration=1.582738497 podStartE2EDuration="4.19053683s" totalImagesPullingTime="2.607798333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:05:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:40.189849576 +0000 UTC m=+54.867812714" watchObservedRunningTime="2026-09-08 07:05:40.19053683 +0000 UTC m=+54.868499946" Sep 08 07:05:40.193671 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.193640 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h"] Sep 08 07:05:40.195716 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.195699 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.198073 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.198052 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5hh8t\"" Sep 08 07:05:40.198299 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.198279 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 08 07:05:40.198452 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.198427 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 08 07:05:40.198529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.198464 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 08 07:05:40.198529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.198471 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 08 07:05:40.198633 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.198427 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 08 07:05:40.206095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.206074 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h"] Sep 08 07:05:40.216360 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.216327 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk"] Sep 08 07:05:40.218907 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.218886 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.221430 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.221398 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 08 07:05:40.221630 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.221611 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 08 07:05:40.221718 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.221698 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fwslk\"" Sep 08 07:05:40.221917 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.221897 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 08 07:05:40.225962 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.225943 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-v7cql"] Sep 08 07:05:40.229284 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.228477 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.229866 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.229839 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk"] Sep 08 07:05:40.230857 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.230833 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 08 07:05:40.230857 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.230841 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vfkd4\"" Sep 08 07:05:40.231005 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.230960 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 08 07:05:40.231298 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.231283 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 08 07:05:40.286024 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.285992 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-wtmp\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.286332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286032 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-accelerators-collector-config\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.286332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286059 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c1072ac4-aa72-492a-bf06-3a0f82c4f522-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.286332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286081 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1072ac4-aa72-492a-bf06-3a0f82c4f522-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.286332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286200 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.286332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286254 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-metrics-client-ca\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.286332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286271 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.286332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286289 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f44eadf3-c51c-418e-b9ef-22266ddf6163-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.286332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286307 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-sys\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.286332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286322 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-root\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.286332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286338 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-textfile\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.286734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286352 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb4xf\" (UniqueName: \"kubernetes.io/projected/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-api-access-gb4xf\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.286734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286383 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f44eadf3-c51c-418e-b9ef-22266ddf6163-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.286734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286477 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hcv4\" (UniqueName: \"kubernetes.io/projected/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-kube-api-access-6hcv4\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.286734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286515 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-tls\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.286734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286533 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f44eadf3-c51c-418e-b9ef-22266ddf6163-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.286734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286549 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxmr5\" (UniqueName: \"kubernetes.io/projected/f44eadf3-c51c-418e-b9ef-22266ddf6163-kube-api-access-kxmr5\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.286734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286573 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.286734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.286625 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.387151 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387052 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.387151 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387098 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-wtmp\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.387151 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387119 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-accelerators-collector-config\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.387151 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387154 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c1072ac4-aa72-492a-bf06-3a0f82c4f522-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387173 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1072ac4-aa72-492a-bf06-3a0f82c4f522-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387286 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387298 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-wtmp\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387315 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-metrics-client-ca\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387352 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387375 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f44eadf3-c51c-418e-b9ef-22266ddf6163-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387396 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-sys\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387423 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-root\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387445 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-textfile\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387466 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb4xf\" (UniqueName: \"kubernetes.io/projected/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-api-access-gb4xf\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.387513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387493 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f44eadf3-c51c-418e-b9ef-22266ddf6163-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387536 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hcv4\" (UniqueName: \"kubernetes.io/projected/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-kube-api-access-6hcv4\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387579 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-tls\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387601 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c1072ac4-aa72-492a-bf06-3a0f82c4f522-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:40.387692 2571 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:05:40.387753 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f44eadf3-c51c-418e-b9ef-22266ddf6163-openshift-state-metrics-tls podName:f44eadf3-c51c-418e-b9ef-22266ddf6163 nodeName:}" failed. No retries permitted until 2026-09-08 07:05:40.88773469 +0000 UTC m=+55.565697791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f44eadf3-c51c-418e-b9ef-22266ddf6163-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-c6l2h" (UID: "f44eadf3-c51c-418e-b9ef-22266ddf6163") : secret "openshift-state-metrics-tls" not found Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387852 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-accelerators-collector-config\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387604 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f44eadf3-c51c-418e-b9ef-22266ddf6163-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387901 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-metrics-client-ca\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387916 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxmr5\" (UniqueName: \"kubernetes.io/projected/f44eadf3-c51c-418e-b9ef-22266ddf6163-kube-api-access-kxmr5\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387949 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.387954 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1072ac4-aa72-492a-bf06-3a0f82c4f522-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.388032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.388034 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-sys\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.388639 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.388039 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-root\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.388639 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.388057 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.388639 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.388139 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-textfile\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.388847 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.388825 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f44eadf3-c51c-418e-b9ef-22266ddf6163-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.390314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.390290 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.390424 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.390399 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.390503 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.390489 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-node-exporter-tls\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.390604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.390588 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.390762 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.390740 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f44eadf3-c51c-418e-b9ef-22266ddf6163-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.395925 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.395899 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxmr5\" (UniqueName: \"kubernetes.io/projected/f44eadf3-c51c-418e-b9ef-22266ddf6163-kube-api-access-kxmr5\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.396014 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.395946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb4xf\" (UniqueName: \"kubernetes.io/projected/c1072ac4-aa72-492a-bf06-3a0f82c4f522-kube-api-access-gb4xf\") pod \"kube-state-metrics-74cb57df7-8lkjk\" (UID: \"c1072ac4-aa72-492a-bf06-3a0f82c4f522\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.396126 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.396108 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hcv4\" (UniqueName: \"kubernetes.io/projected/71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec-kube-api-access-6hcv4\") pod \"node-exporter-v7cql\" (UID: \"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec\") " pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.529434 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.529403 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" Sep 08 07:05:40.537372 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.537345 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v7cql" Sep 08 07:05:40.546993 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:40.546965 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e3d7cd_db7c_4757_9a8d_d42d4d7e2cec.slice/crio-83a89928cf9b5625bc1ba90af5c327ccccb7e2e26834ffb37bba6fcecca6e495 WatchSource:0}: Error finding container 83a89928cf9b5625bc1ba90af5c327ccccb7e2e26834ffb37bba6fcecca6e495: Status 404 returned error can't find the container with id 83a89928cf9b5625bc1ba90af5c327ccccb7e2e26834ffb37bba6fcecca6e495 Sep 08 07:05:40.801901 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.801876 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk"] Sep 08 07:05:40.803942 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:40.803912 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1072ac4_aa72_492a_bf06_3a0f82c4f522.slice/crio-6c804638b0f8b337b5f2d5fba8bae49beb1944b98790457ec6a98cef06cae8a0 WatchSource:0}: Error finding container 6c804638b0f8b337b5f2d5fba8bae49beb1944b98790457ec6a98cef06cae8a0: Status 404 returned error can't find the container with id 6c804638b0f8b337b5f2d5fba8bae49beb1944b98790457ec6a98cef06cae8a0 Sep 08 07:05:40.891437 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.891403 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f44eadf3-c51c-418e-b9ef-22266ddf6163-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:40.893736 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:40.893715 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f44eadf3-c51c-418e-b9ef-22266ddf6163-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-c6l2h\" (UID: \"f44eadf3-c51c-418e-b9ef-22266ddf6163\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:41.105464 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:41.105379 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" Sep 08 07:05:41.169566 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:41.169526 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" event={"ID":"c1072ac4-aa72-492a-bf06-3a0f82c4f522","Type":"ContainerStarted","Data":"6c804638b0f8b337b5f2d5fba8bae49beb1944b98790457ec6a98cef06cae8a0"} Sep 08 07:05:41.170667 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:41.170647 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v7cql" event={"ID":"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec","Type":"ContainerStarted","Data":"83a89928cf9b5625bc1ba90af5c327ccccb7e2e26834ffb37bba6fcecca6e495"} Sep 08 07:05:41.375807 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:41.375716 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h"] Sep 08 07:05:41.501794 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:41.501758 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf44eadf3_c51c_418e_b9ef_22266ddf6163.slice/crio-3cf9fc15d96fc2446f03c3932a466bb3c25ce01b98f7f826dbe4ed8b066df520 WatchSource:0}: Error finding container 3cf9fc15d96fc2446f03c3932a466bb3c25ce01b98f7f826dbe4ed8b066df520: Status 404 returned error can't find the container with id 3cf9fc15d96fc2446f03c3932a466bb3c25ce01b98f7f826dbe4ed8b066df520 Sep 08 07:05:42.180088 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:42.180047 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" event={"ID":"f44eadf3-c51c-418e-b9ef-22266ddf6163","Type":"ContainerStarted","Data":"6a8a544eec41a0c84b895b2baf2a559fd851ef2919c85d4b292307420fca1170"} Sep 08 07:05:42.181026 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:42.180830 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" event={"ID":"f44eadf3-c51c-418e-b9ef-22266ddf6163","Type":"ContainerStarted","Data":"c24de0f6f618b70ca0d23f97d5fef0f50cc63eb34dcb2e491658e20e905a665e"} Sep 08 07:05:42.181026 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:42.180874 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" event={"ID":"f44eadf3-c51c-418e-b9ef-22266ddf6163","Type":"ContainerStarted","Data":"3cf9fc15d96fc2446f03c3932a466bb3c25ce01b98f7f826dbe4ed8b066df520"} Sep 08 07:05:42.182844 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:42.182243 2571 generic.go:356] "Generic (PLEG): container finished" podID="71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec" containerID="9c2fa2ab7ad8b57e27d7319c93d3e1e4084d748a8e43b298013ab2fe282aaff3" exitCode=0 Sep 08 07:05:42.182844 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:42.182345 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v7cql" event={"ID":"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec","Type":"ContainerDied","Data":"9c2fa2ab7ad8b57e27d7319c93d3e1e4084d748a8e43b298013ab2fe282aaff3"} Sep 08 07:05:42.185731 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:42.185689 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" event={"ID":"c1072ac4-aa72-492a-bf06-3a0f82c4f522","Type":"ContainerStarted","Data":"c310152eca84af5951965e6aa96589648442800407d7b5e170b99b08430d440c"} Sep 08 07:05:43.192192 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:43.192096 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v7cql" event={"ID":"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec","Type":"ContainerStarted","Data":"840f658da2a8c75a6678a2d289de70cc372c4db2340d9e444bf46733c46abc36"} Sep 08 07:05:43.192192 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:43.192159 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v7cql" event={"ID":"71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec","Type":"ContainerStarted","Data":"9379f3490c70637887ddde59cd7374cf7611a5eabe8adb3be8300275affebd7e"} Sep 08 07:05:43.197763 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:43.194294 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" event={"ID":"c1072ac4-aa72-492a-bf06-3a0f82c4f522","Type":"ContainerStarted","Data":"981f2873d984b45127c6669e41be34bf9e9829d93afd7b081a064109f2a5a753"} Sep 08 07:05:43.197763 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:43.194777 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" event={"ID":"c1072ac4-aa72-492a-bf06-3a0f82c4f522","Type":"ContainerStarted","Data":"0a638bfa973cff218ddf5d45ec80f90ca948468e0cf91ca73d212b297a965c95"} Sep 08 07:05:43.200530 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:43.200504 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" event={"ID":"f44eadf3-c51c-418e-b9ef-22266ddf6163","Type":"ContainerStarted","Data":"63e202c1b9a34663604e79b85584551a1528897799ab478de3ac06412aa0a088"} Sep 08 07:05:43.218178 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:43.218128 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-v7cql" podStartSLOduration=2.157299571 podStartE2EDuration="3.218115033s" totalImagesPullingTime="978.241425ms" totalInitContainerRuntime="82.574037ms" isStatefulPod=true podCreationTimestamp="2026-09-08 07:05:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:43.216624574 +0000 UTC m=+57.894587710" watchObservedRunningTime="2026-09-08 07:05:43.218115033 +0000 UTC m=+57.896078153" Sep 08 07:05:43.238576 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:43.238525 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-c6l2h" podStartSLOduration=2.201446287 podStartE2EDuration="3.238486791s" totalImagesPullingTime="1.037040504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:05:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:43.238340004 +0000 UTC m=+57.916303141" watchObservedRunningTime="2026-09-08 07:05:43.238486791 +0000 UTC m=+57.916449888" Sep 08 07:05:43.255267 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:43.255191 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8lkjk" podStartSLOduration=1.990104313 podStartE2EDuration="3.255174693s" totalImagesPullingTime="1.26507038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:05:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:43.254653205 +0000 UTC m=+57.932616340" watchObservedRunningTime="2026-09-08 07:05:43.255174693 +0000 UTC m=+57.933137847" Sep 08 07:05:44.989729 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:44.989691 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs"] Sep 08 07:05:44.992649 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:44.992627 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" Sep 08 07:05:44.994907 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:44.994880 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 08 07:05:44.995019 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:44.994922 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6mshm\"" Sep 08 07:05:45.000417 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:45.000380 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs"] Sep 08 07:05:45.024853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:45.024822 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mch95\" (UniqueName: \"kubernetes.io/projected/3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa-kube-api-access-mch95\") pod \"monitoring-plugin-85bbbbbf4-4sgbs\" (UID: \"3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" Sep 08 07:05:45.025026 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:45.024872 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-4sgbs\" (UID: \"3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" Sep 08 07:05:45.125453 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:45.125418 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-4sgbs\" (UID: \"3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" Sep 08 07:05:45.125617 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:45.125500 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mch95\" (UniqueName: \"kubernetes.io/projected/3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa-kube-api-access-mch95\") pod \"monitoring-plugin-85bbbbbf4-4sgbs\" (UID: \"3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" Sep 08 07:05:45.127815 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:45.127786 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-4sgbs\" (UID: \"3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" Sep 08 07:05:45.134051 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:45.134024 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mch95\" (UniqueName: \"kubernetes.io/projected/3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa-kube-api-access-mch95\") pod \"monitoring-plugin-85bbbbbf4-4sgbs\" (UID: \"3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" Sep 08 07:05:45.301746 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:45.301655 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" Sep 08 07:05:45.572081 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:45.570320 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs"] Sep 08 07:05:45.576261 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:45.576224 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d9f7bc4_c7a2_4268_b7a1_dcd247b58efa.slice/crio-1c4357d533e219f24c7c5d54c193d46f08fb3d8dba899400a14fd2aa0ef8f723 WatchSource:0}: Error finding container 1c4357d533e219f24c7c5d54c193d46f08fb3d8dba899400a14fd2aa0ef8f723: Status 404 returned error can't find the container with id 1c4357d533e219f24c7c5d54c193d46f08fb3d8dba899400a14fd2aa0ef8f723 Sep 08 07:05:46.079122 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:46.079094 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsrt2" Sep 08 07:05:46.212275 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:46.212232 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" event={"ID":"3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa","Type":"ContainerStarted","Data":"1c4357d533e219f24c7c5d54c193d46f08fb3d8dba899400a14fd2aa0ef8f723"} Sep 08 07:05:46.416461 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:46.416377 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:46.416461 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:46.416449 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:46.421701 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:46.421674 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:47.118908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:47.118885 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bf64cffd6-qdfp2" Sep 08 07:05:47.219341 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:47.219305 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" event={"ID":"3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa","Type":"ContainerStarted","Data":"d1ef127822e540a35ac672a88203cf33d36a81270773ee4101fd976628279317"} Sep 08 07:05:47.223537 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:47.223509 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:05:47.234837 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:47.234784 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" podStartSLOduration=1.716871415 podStartE2EDuration="3.234763753s" totalImagesPullingTime="1.517892338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:47.233849535 +0000 UTC m=+61.911812676" watchObservedRunningTime="2026-09-08 07:05:47.234763753 +0000 UTC m=+61.912726870" Sep 08 07:05:48.223041 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:48.223005 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" Sep 08 07:05:48.227444 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:48.227423 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-4sgbs" Sep 08 07:05:51.475537 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.475506 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:51.478494 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.478466 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 07:05:51.487771 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.487748 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b282970a-79b3-42c5-8200-192bf1b0ac43-metrics-certs\") pod \"network-metrics-daemon-8k4kq\" (UID: \"b282970a-79b3-42c5-8200-192bf1b0ac43\") " pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:51.576850 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.576814 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:51.579478 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.579455 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 07:05:51.590027 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.589990 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43-original-pull-secret\") pod \"global-pull-secret-syncer-gj26q\" (UID: \"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43\") " pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:51.638435 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.638404 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ldq7w\"" Sep 08 07:05:51.646518 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.646495 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8k4kq" Sep 08 07:05:51.678078 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.678052 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kntkk\" (UniqueName: \"kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk\") pod \"network-check-target-zts7r\" (UID: \"433a278e-8a9d-44ad-972c-f36e39dd23d0\") " pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:51.680621 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.680599 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 07:05:51.690875 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.690852 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 07:05:51.700991 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.700963 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kntkk\" (UniqueName: \"kubernetes.io/projected/433a278e-8a9d-44ad-972c-f36e39dd23d0-kube-api-access-kntkk\") pod \"network-check-target-zts7r\" (UID: \"433a278e-8a9d-44ad-972c-f36e39dd23d0\") " pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:51.716884 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.716860 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gj26q" Sep 08 07:05:51.726466 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.726398 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7mcm7\"" Sep 08 07:05:51.734305 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.734274 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:51.938271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:51.938246 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8k4kq"] Sep 08 07:05:51.940622 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:51.940591 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb282970a_79b3_42c5_8200_192bf1b0ac43.slice/crio-06033cda2f444c4e36d25c26c8cba7e692ffce8cd4e773a78b59eb472c55495e WatchSource:0}: Error finding container 06033cda2f444c4e36d25c26c8cba7e692ffce8cd4e773a78b59eb472c55495e: Status 404 returned error can't find the container with id 06033cda2f444c4e36d25c26c8cba7e692ffce8cd4e773a78b59eb472c55495e Sep 08 07:05:52.005205 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:52.004995 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gj26q"] Sep 08 07:05:52.007497 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:52.007471 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3c3143_6d3e_47ae_a9f5_c72cbcbb8a43.slice/crio-128f577cbd687cf0f671bd6ba68d2cbc0344294d1c2a71638f43d25c85dde4ef WatchSource:0}: Error finding container 128f577cbd687cf0f671bd6ba68d2cbc0344294d1c2a71638f43d25c85dde4ef: Status 404 returned error can't find the container with id 128f577cbd687cf0f671bd6ba68d2cbc0344294d1c2a71638f43d25c85dde4ef Sep 08 07:05:52.033265 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:52.033242 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zts7r"] Sep 08 07:05:52.035275 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:05:52.035252 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod433a278e_8a9d_44ad_972c_f36e39dd23d0.slice/crio-9fa110e06f7f11baddf7339689b796417a15ed18e50f98a59dab42c98a04de77 WatchSource:0}: Error finding container 9fa110e06f7f11baddf7339689b796417a15ed18e50f98a59dab42c98a04de77: Status 404 returned error can't find the container with id 9fa110e06f7f11baddf7339689b796417a15ed18e50f98a59dab42c98a04de77 Sep 08 07:05:52.236884 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:52.236831 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zts7r" event={"ID":"433a278e-8a9d-44ad-972c-f36e39dd23d0","Type":"ContainerStarted","Data":"9fa110e06f7f11baddf7339689b796417a15ed18e50f98a59dab42c98a04de77"} Sep 08 07:05:52.238067 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:52.238028 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gj26q" event={"ID":"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43","Type":"ContainerStarted","Data":"128f577cbd687cf0f671bd6ba68d2cbc0344294d1c2a71638f43d25c85dde4ef"} Sep 08 07:05:52.239106 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:52.239087 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8k4kq" event={"ID":"b282970a-79b3-42c5-8200-192bf1b0ac43","Type":"ContainerStarted","Data":"06033cda2f444c4e36d25c26c8cba7e692ffce8cd4e773a78b59eb472c55495e"} Sep 08 07:05:53.245597 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:53.245556 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8k4kq" event={"ID":"b282970a-79b3-42c5-8200-192bf1b0ac43","Type":"ContainerStarted","Data":"56ce35383c73010cb4c62edfb3957e4a23031e2202ae16b50eefbba52b03ef11"} Sep 08 07:05:53.246057 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:53.245626 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8k4kq" event={"ID":"b282970a-79b3-42c5-8200-192bf1b0ac43","Type":"ContainerStarted","Data":"ae1b7a71cba7dd5cbb9c4b1d78dec9b17a78a64a52ae53957b98c479bced78cb"} Sep 08 07:05:53.267067 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:53.267009 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8k4kq" podStartSLOduration=66.32081997 podStartE2EDuration="1m7.266991715s" totalImagesPullingTime="946.171745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:53.266464318 +0000 UTC m=+67.944427463" watchObservedRunningTime="2026-09-08 07:05:53.266991715 +0000 UTC m=+67.944954832" Sep 08 07:05:56.391509 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:56.391476 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f84dfd9d5-ms5zl"] Sep 08 07:05:57.265356 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:57.265319 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zts7r" event={"ID":"433a278e-8a9d-44ad-972c-f36e39dd23d0","Type":"ContainerStarted","Data":"b7a275c8c18342f08c08eefa19c3159903768a73319fe4cd5dd1cecf7447a53f"} Sep 08 07:05:57.265545 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:57.265451 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:05:57.266931 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:57.266899 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gj26q" event={"ID":"fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43","Type":"ContainerStarted","Data":"37dd47793b2c97aea913e79f93398ba69a14ae9d72ce7faf34db78b40a9e0022"} Sep 08 07:05:57.282892 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:57.282851 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zts7r" podStartSLOduration=66.63417425 podStartE2EDuration="1m11.282837239s" totalImagesPullingTime="4.648662989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:57.282140067 +0000 UTC m=+71.960103184" watchObservedRunningTime="2026-09-08 07:05:57.282837239 +0000 UTC m=+71.960800355" Sep 08 07:05:57.301121 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:05:57.301075 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gj26q" podStartSLOduration=66.621207913 podStartE2EDuration="1m11.301061067s" totalImagesPullingTime="4.679853154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:05:57.30038883 +0000 UTC m=+71.978351937" watchObservedRunningTime="2026-09-08 07:05:57.301061067 +0000 UTC m=+71.979024183" Sep 08 07:06:21.413625 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.413505 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f84dfd9d5-ms5zl" podUID="ed578ab9-2f2f-4c21-abb4-6effbd18011d" containerName="console" containerID="cri-o://1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55" gracePeriod=15 Sep 08 07:06:21.643535 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.643512 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:06:21.795839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.795803 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlcjz\" (UniqueName: \"kubernetes.io/projected/ed578ab9-2f2f-4c21-abb4-6effbd18011d-kube-api-access-xlcjz\") pod \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " Sep 08 07:06:21.796011 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.795877 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-service-ca\") pod \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " Sep 08 07:06:21.796011 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.795973 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-trusted-ca-bundle\") pod \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " Sep 08 07:06:21.796120 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.796014 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-serving-cert\") pod \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " Sep 08 07:06:21.796120 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.796039 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-config\") pod \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " Sep 08 07:06:21.796120 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.796060 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-oauth-config\") pod \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " Sep 08 07:06:21.796120 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.796086 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-oauth-serving-cert\") pod \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\" (UID: \"ed578ab9-2f2f-4c21-abb4-6effbd18011d\") " Sep 08 07:06:21.796394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.796371 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-service-ca" (OuterVolumeSpecName: "service-ca") pod "ed578ab9-2f2f-4c21-abb4-6effbd18011d" (UID: "ed578ab9-2f2f-4c21-abb4-6effbd18011d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:06:21.796517 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.796487 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ed578ab9-2f2f-4c21-abb4-6effbd18011d" (UID: "ed578ab9-2f2f-4c21-abb4-6effbd18011d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:06:21.796611 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.796529 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-config" (OuterVolumeSpecName: "console-config") pod "ed578ab9-2f2f-4c21-abb4-6effbd18011d" (UID: "ed578ab9-2f2f-4c21-abb4-6effbd18011d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:06:21.796611 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.796583 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ed578ab9-2f2f-4c21-abb4-6effbd18011d" (UID: "ed578ab9-2f2f-4c21-abb4-6effbd18011d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:06:21.798293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.798266 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed578ab9-2f2f-4c21-abb4-6effbd18011d-kube-api-access-xlcjz" (OuterVolumeSpecName: "kube-api-access-xlcjz") pod "ed578ab9-2f2f-4c21-abb4-6effbd18011d" (UID: "ed578ab9-2f2f-4c21-abb4-6effbd18011d"). InnerVolumeSpecName "kube-api-access-xlcjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:06:21.798398 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.798308 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ed578ab9-2f2f-4c21-abb4-6effbd18011d" (UID: "ed578ab9-2f2f-4c21-abb4-6effbd18011d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:06:21.798398 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.798362 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ed578ab9-2f2f-4c21-abb4-6effbd18011d" (UID: "ed578ab9-2f2f-4c21-abb4-6effbd18011d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:06:21.897171 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.897132 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:06:21.897171 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.897158 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-trusted-ca-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:06:21.897171 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.897168 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-serving-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:06:21.897171 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.897178 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-config\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:06:21.897428 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.897193 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed578ab9-2f2f-4c21-abb4-6effbd18011d-console-oauth-config\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:06:21.897428 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.897205 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed578ab9-2f2f-4c21-abb4-6effbd18011d-oauth-serving-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:06:21.897428 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:21.897229 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlcjz\" (UniqueName: \"kubernetes.io/projected/ed578ab9-2f2f-4c21-abb4-6effbd18011d-kube-api-access-xlcjz\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:06:22.350326 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:22.350291 2571 generic.go:356] "Generic (PLEG): container finished" podID="ed578ab9-2f2f-4c21-abb4-6effbd18011d" containerID="1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55" exitCode=0 Sep 08 07:06:22.350490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:22.350337 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f84dfd9d5-ms5zl" Sep 08 07:06:22.350490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:22.350347 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f84dfd9d5-ms5zl" event={"ID":"ed578ab9-2f2f-4c21-abb4-6effbd18011d","Type":"ContainerDied","Data":"1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55"} Sep 08 07:06:22.350490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:22.350387 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f84dfd9d5-ms5zl" event={"ID":"ed578ab9-2f2f-4c21-abb4-6effbd18011d","Type":"ContainerDied","Data":"f357b10d393b01dfa4dc5173d3a103a683ad87bcaa27c2e3fd50cf6583b023ee"} Sep 08 07:06:22.350490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:22.350405 2571 scope.go:117] "RemoveContainer" containerID="1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55" Sep 08 07:06:22.358425 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:22.358403 2571 scope.go:117] "RemoveContainer" containerID="1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55" Sep 08 07:06:22.358702 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:06:22.358679 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55\": container with ID starting with 1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55 not found: ID does not exist" containerID="1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55" Sep 08 07:06:22.358767 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:22.358708 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55"} err="failed to get container status \"1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55\": rpc error: code = NotFound desc = could not find container \"1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55\": container with ID starting with 1b6525826a340167bafc48d5dbb34767461fef24c2e1fadacbd188139f05ca55 not found: ID does not exist" Sep 08 07:06:22.367959 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:22.367934 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f84dfd9d5-ms5zl"] Sep 08 07:06:22.370010 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:22.369992 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f84dfd9d5-ms5zl"] Sep 08 07:06:23.807384 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:23.807350 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed578ab9-2f2f-4c21-abb4-6effbd18011d" path="/var/lib/kubelet/pods/ed578ab9-2f2f-4c21-abb4-6effbd18011d/volumes" Sep 08 07:06:28.273182 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:06:28.273151 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zts7r" Sep 08 07:07:02.673122 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.673084 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 07:07:02.673604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.673380 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed578ab9-2f2f-4c21-abb4-6effbd18011d" containerName="console" Sep 08 07:07:02.673604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.673393 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed578ab9-2f2f-4c21-abb4-6effbd18011d" containerName="console" Sep 08 07:07:02.673604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.673444 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed578ab9-2f2f-4c21-abb4-6effbd18011d" containerName="console" Sep 08 07:07:02.676915 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.676894 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.680396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.680361 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4rknp\"" Sep 08 07:07:02.680652 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.680625 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 07:07:02.680863 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.680842 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 07:07:02.680994 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.680969 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d255f57e-7e27-46b5-8900-15b3d01208ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681014 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 07:07:02.681069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681031 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681061 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d255f57e-7e27-46b5-8900-15b3d01208ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681075 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 07:07:02.681254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681094 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681128 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681174 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681233 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681498 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681279 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mmxv\" (UniqueName: \"kubernetes.io/projected/d255f57e-7e27-46b5-8900-15b3d01208ea-kube-api-access-6mmxv\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681498 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681317 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 07:07:02.681498 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681425 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d255f57e-7e27-46b5-8900-15b3d01208ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681498 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681486 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d255f57e-7e27-46b5-8900-15b3d01208ea-config-out\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681678 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681546 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681678 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681579 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-web-config\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681678 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681643 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d255f57e-7e27-46b5-8900-15b3d01208ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.681893 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681873 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 07:07:02.682402 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.681583 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 07:07:02.683429 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.683402 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 07:07:02.686956 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.686924 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 07:07:02.689172 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.689153 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 07:07:02.783449 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783413 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783449 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783451 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783475 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783492 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783515 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mmxv\" (UniqueName: \"kubernetes.io/projected/d255f57e-7e27-46b5-8900-15b3d01208ea-kube-api-access-6mmxv\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783536 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d255f57e-7e27-46b5-8900-15b3d01208ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783552 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d255f57e-7e27-46b5-8900-15b3d01208ea-config-out\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783567 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783591 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-web-config\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783622 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d255f57e-7e27-46b5-8900-15b3d01208ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783651 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d255f57e-7e27-46b5-8900-15b3d01208ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783685 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.783706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.783710 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d255f57e-7e27-46b5-8900-15b3d01208ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.784553 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.784527 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d255f57e-7e27-46b5-8900-15b3d01208ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.784984 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.784962 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d255f57e-7e27-46b5-8900-15b3d01208ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.786608 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.786583 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.786701 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.786681 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.786755 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.786681 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.786941 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.786911 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d255f57e-7e27-46b5-8900-15b3d01208ea-config-out\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.787125 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.787077 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.787241 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.787184 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-web-config\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.787241 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.787185 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.787521 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.787501 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d255f57e-7e27-46b5-8900-15b3d01208ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.787908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.787888 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d255f57e-7e27-46b5-8900-15b3d01208ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.788871 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.788853 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d255f57e-7e27-46b5-8900-15b3d01208ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.791796 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.791778 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mmxv\" (UniqueName: \"kubernetes.io/projected/d255f57e-7e27-46b5-8900-15b3d01208ea-kube-api-access-6mmxv\") pod \"alertmanager-main-0\" (UID: \"d255f57e-7e27-46b5-8900-15b3d01208ea\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:02.989583 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:02.989549 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 07:07:03.265334 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:03.265305 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 07:07:03.266552 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:07:03.266526 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd255f57e_7e27_46b5_8900_15b3d01208ea.slice/crio-70cbd0f5e343dad251d19a8a1cb2a524bab0eaa1ef7b67aee950e4503e9b5bea WatchSource:0}: Error finding container 70cbd0f5e343dad251d19a8a1cb2a524bab0eaa1ef7b67aee950e4503e9b5bea: Status 404 returned error can't find the container with id 70cbd0f5e343dad251d19a8a1cb2a524bab0eaa1ef7b67aee950e4503e9b5bea Sep 08 07:07:03.496327 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:03.496295 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d255f57e-7e27-46b5-8900-15b3d01208ea","Type":"ContainerStarted","Data":"70cbd0f5e343dad251d19a8a1cb2a524bab0eaa1ef7b67aee950e4503e9b5bea"} Sep 08 07:07:04.535139 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.535113 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b"] Sep 08 07:07:04.537399 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.537380 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.541449 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.541427 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 08 07:07:04.541529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.541427 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dxrcr\"" Sep 08 07:07:04.542255 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.542237 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 08 07:07:04.542344 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.542267 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 08 07:07:04.542344 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.542239 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 08 07:07:04.542909 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.542885 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 08 07:07:04.557787 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.557765 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 08 07:07:04.579121 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.579084 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b"] Sep 08 07:07:04.595141 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.595111 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-telemeter-client-tls\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.595348 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.595154 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.595348 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.595275 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-metrics-client-ca\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.595502 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.595366 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-serving-certs-ca-bundle\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.595502 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.595391 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.595581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.595502 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxg8m\" (UniqueName: \"kubernetes.io/projected/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-kube-api-access-kxg8m\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.595581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.595566 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-federate-client-tls\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.595671 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.595596 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-secret-telemeter-client\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.696497 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.696461 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-secret-telemeter-client\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.696497 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.696503 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-telemeter-client-tls\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.696710 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.696528 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.696710 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.696557 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-metrics-client-ca\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.696710 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.696589 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-serving-certs-ca-bundle\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.696710 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.696613 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.696710 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.696684 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxg8m\" (UniqueName: \"kubernetes.io/projected/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-kube-api-access-kxg8m\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.696917 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.696729 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-federate-client-tls\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.697448 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.697418 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-metrics-client-ca\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.697448 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.697439 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.697640 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.697429 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-serving-certs-ca-bundle\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.699126 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.699091 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.699288 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.699267 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-secret-telemeter-client\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.699506 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.699491 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-telemeter-client-tls\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.699628 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.699610 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-federate-client-tls\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.705099 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.705079 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxg8m\" (UniqueName: \"kubernetes.io/projected/3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7-kube-api-access-kxg8m\") pod \"telemeter-client-7fc5f687c4-nbb2b\" (UID: \"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7\") " pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:04.849709 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:04.849630 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" Sep 08 07:07:05.127121 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:05.126462 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b"] Sep 08 07:07:05.129571 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:07:05.129528 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec9ebfd_c95c_49d3_9df1_3f20ca8c2ad7.slice/crio-d2dc0f04f3e88429d284c810580cc07954dfc4833f20d5e770e18d5efea8562e WatchSource:0}: Error finding container d2dc0f04f3e88429d284c810580cc07954dfc4833f20d5e770e18d5efea8562e: Status 404 returned error can't find the container with id d2dc0f04f3e88429d284c810580cc07954dfc4833f20d5e770e18d5efea8562e Sep 08 07:07:05.504388 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:05.504349 2571 generic.go:356] "Generic (PLEG): container finished" podID="d255f57e-7e27-46b5-8900-15b3d01208ea" containerID="a9b61b0c19f9f70e1f3d450310ca4a2022f78dfd7ab09f4c0b1b3ebd83ce84a3" exitCode=0 Sep 08 07:07:05.504533 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:05.504429 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d255f57e-7e27-46b5-8900-15b3d01208ea","Type":"ContainerDied","Data":"a9b61b0c19f9f70e1f3d450310ca4a2022f78dfd7ab09f4c0b1b3ebd83ce84a3"} Sep 08 07:07:05.505743 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:05.505720 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" event={"ID":"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7","Type":"ContainerStarted","Data":"d2dc0f04f3e88429d284c810580cc07954dfc4833f20d5e770e18d5efea8562e"} Sep 08 07:07:07.515562 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:07.515532 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" event={"ID":"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7","Type":"ContainerStarted","Data":"41c970959c132a896979f30634250586ad681998fa920643bd8750f1fa5f594a"} Sep 08 07:07:07.518097 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:07.517985 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d255f57e-7e27-46b5-8900-15b3d01208ea","Type":"ContainerStarted","Data":"d291c5fec34428ea10010dca67d6ea3f1319efc6366f0b0ed0fae58f193a44be"} Sep 08 07:07:08.524801 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.524759 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" event={"ID":"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7","Type":"ContainerStarted","Data":"af5505dbe4796cbb6d3a0d0f2a7d2ea6fb50b764eb90234a4acaaa064c474495"} Sep 08 07:07:08.525388 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.524834 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" event={"ID":"3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7","Type":"ContainerStarted","Data":"a4324766e01b43056def25dd7ab970158122b2c50e9da5c9870927ce1c6f3636"} Sep 08 07:07:08.528083 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.528052 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d255f57e-7e27-46b5-8900-15b3d01208ea","Type":"ContainerStarted","Data":"77463fbb07dd9c9bba956f31820ea3786e769a1acb310cc55658254d759805e4"} Sep 08 07:07:08.528254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.528115 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d255f57e-7e27-46b5-8900-15b3d01208ea","Type":"ContainerStarted","Data":"1c0a0d22db5375d9ee5981d701504b743f35a957ab1066cba40c945fd0df86b3"} Sep 08 07:07:08.528254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.528141 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d255f57e-7e27-46b5-8900-15b3d01208ea","Type":"ContainerStarted","Data":"923f77a0ec8b22664efc73650f25d3d2cece090fe1718ec6f5312f48ab25eff0"} Sep 08 07:07:08.528254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.528164 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d255f57e-7e27-46b5-8900-15b3d01208ea","Type":"ContainerStarted","Data":"c5427150df8f263cf8ac46e2a29eb1bd523dd71013306e464c2dfc12c66f6ed7"} Sep 08 07:07:08.549439 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.549388 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7fc5f687c4-nbb2b" podStartSLOduration=2.275376079 podStartE2EDuration="4.549371538s" totalImagesPullingTime="2.273995459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:07:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:07:08.547802137 +0000 UTC m=+143.225765301" watchObservedRunningTime="2026-09-08 07:07:08.549371538 +0000 UTC m=+143.227334653" Sep 08 07:07:08.633977 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.633945 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d5b45f99d-9zr2z"] Sep 08 07:07:08.636301 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.636282 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.639003 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.638981 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 08 07:07:08.639312 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.639288 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mxlnj\"" Sep 08 07:07:08.639430 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.639328 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 08 07:07:08.639565 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.639546 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 08 07:07:08.639833 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.639815 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 08 07:07:08.639923 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.639834 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 08 07:07:08.639923 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.639815 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 08 07:07:08.640038 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.640017 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 08 07:07:08.644371 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.644245 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 08 07:07:08.647623 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.647601 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d5b45f99d-9zr2z"] Sep 08 07:07:08.733002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.732977 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-serving-cert\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.733129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.733016 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-console-config\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.733129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.733037 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tpfq\" (UniqueName: \"kubernetes.io/projected/ec0cac89-3da1-4c31-8713-06d09d19092b-kube-api-access-6tpfq\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.733129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.733087 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-trusted-ca-bundle\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.733306 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.733142 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-oauth-config\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.733306 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.733164 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-service-ca\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.733306 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.733181 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-oauth-serving-cert\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.834311 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.834209 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-serving-cert\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.834311 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.834285 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-console-config\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.834311 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.834308 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpfq\" (UniqueName: \"kubernetes.io/projected/ec0cac89-3da1-4c31-8713-06d09d19092b-kube-api-access-6tpfq\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.834579 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.834327 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-trusted-ca-bundle\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.834579 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.834388 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-oauth-config\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.834579 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.834410 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-service-ca\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.834730 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.834636 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-oauth-serving-cert\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.835136 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.835109 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-service-ca\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.835280 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.835109 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-console-config\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.835280 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.835236 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-oauth-serving-cert\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.835598 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.835578 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-trusted-ca-bundle\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.837048 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.837025 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-serving-cert\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.837171 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.837149 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-oauth-config\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.842692 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.842672 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tpfq\" (UniqueName: \"kubernetes.io/projected/ec0cac89-3da1-4c31-8713-06d09d19092b-kube-api-access-6tpfq\") pod \"console-d5b45f99d-9zr2z\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:08.947581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:08.947540 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:09.218770 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:09.218713 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d5b45f99d-9zr2z"] Sep 08 07:07:09.220876 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:07:09.220853 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec0cac89_3da1_4c31_8713_06d09d19092b.slice/crio-de677007c1705a2a8141e51a3053a98d7941b78cbbac1c6f0cf55f1c54bdbcb2 WatchSource:0}: Error finding container de677007c1705a2a8141e51a3053a98d7941b78cbbac1c6f0cf55f1c54bdbcb2: Status 404 returned error can't find the container with id de677007c1705a2a8141e51a3053a98d7941b78cbbac1c6f0cf55f1c54bdbcb2 Sep 08 07:07:09.533373 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:09.533265 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5b45f99d-9zr2z" event={"ID":"ec0cac89-3da1-4c31-8713-06d09d19092b","Type":"ContainerStarted","Data":"5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4"} Sep 08 07:07:09.533373 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:09.533328 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5b45f99d-9zr2z" event={"ID":"ec0cac89-3da1-4c31-8713-06d09d19092b","Type":"ContainerStarted","Data":"de677007c1705a2a8141e51a3053a98d7941b78cbbac1c6f0cf55f1c54bdbcb2"} Sep 08 07:07:09.536401 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:09.536369 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d255f57e-7e27-46b5-8900-15b3d01208ea","Type":"ContainerStarted","Data":"de8bef54a42a1230afd697c80c24ca4c5a5debbfc6727717e14ec34e77893961"} Sep 08 07:07:09.551471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:09.551424 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d5b45f99d-9zr2z" podStartSLOduration=1.55140918 podStartE2EDuration="1.55140918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:07:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:07:09.550398233 +0000 UTC m=+144.228361578" watchObservedRunningTime="2026-09-08 07:07:09.55140918 +0000 UTC m=+144.229372299" Sep 08 07:07:09.574961 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:09.574912 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.459070774 podStartE2EDuration="7.574896792s" totalImagesPullingTime="4.094408053s" totalInitContainerRuntime="21.417965ms" isStatefulPod=false podCreationTimestamp="2026-09-08 07:07:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:07:09.573030184 +0000 UTC m=+144.250993319" watchObservedRunningTime="2026-09-08 07:07:09.574896792 +0000 UTC m=+144.252859908" Sep 08 07:07:18.948138 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:18.948078 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:18.948580 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:18.948161 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:18.952930 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:18.952899 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:07:19.581258 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:07:19.581225 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:08:18.377567 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.377531 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bd9f9987c-9c2q8"] Sep 08 07:08:18.380932 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.380911 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.392826 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.392709 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bd9f9987c-9c2q8"] Sep 08 07:08:18.472094 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.472055 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-oauth-serving-cert\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.472279 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.472101 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-oauth-config\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.472279 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.472129 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt2q7\" (UniqueName: \"kubernetes.io/projected/3ccb3186-601f-467c-b51b-99639536e65a-kube-api-access-tt2q7\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.472279 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.472154 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-trusted-ca-bundle\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.472279 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.472173 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-console-config\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.472279 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.472188 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-service-ca\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.472496 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.472282 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-serving-cert\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.573501 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.573467 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt2q7\" (UniqueName: \"kubernetes.io/projected/3ccb3186-601f-467c-b51b-99639536e65a-kube-api-access-tt2q7\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.573501 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.573505 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-trusted-ca-bundle\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.573696 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.573534 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-console-config\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.573696 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.573556 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-service-ca\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.573696 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.573593 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-serving-cert\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.573696 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.573636 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-oauth-serving-cert\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.573696 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.573681 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-oauth-config\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.574327 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.574296 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-console-config\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.574442 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.574369 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-service-ca\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.574442 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.574433 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-oauth-serving-cert\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.574519 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.574470 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-trusted-ca-bundle\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.576724 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.576699 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-oauth-config\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.576816 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.576711 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-serving-cert\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.581266 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.581245 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt2q7\" (UniqueName: \"kubernetes.io/projected/3ccb3186-601f-467c-b51b-99639536e65a-kube-api-access-tt2q7\") pod \"console-6bd9f9987c-9c2q8\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.689747 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.689656 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:18.959405 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:18.959373 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bd9f9987c-9c2q8"] Sep 08 07:08:18.963542 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:08:18.963518 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ccb3186_601f_467c_b51b_99639536e65a.slice/crio-8f5c600814399d8a3438712091f5309fc49ee3d0827f87b520bcb8e88017d60c WatchSource:0}: Error finding container 8f5c600814399d8a3438712091f5309fc49ee3d0827f87b520bcb8e88017d60c: Status 404 returned error can't find the container with id 8f5c600814399d8a3438712091f5309fc49ee3d0827f87b520bcb8e88017d60c Sep 08 07:08:19.795665 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:19.795626 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bd9f9987c-9c2q8" event={"ID":"3ccb3186-601f-467c-b51b-99639536e65a","Type":"ContainerStarted","Data":"150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244"} Sep 08 07:08:19.796130 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:19.795684 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bd9f9987c-9c2q8" event={"ID":"3ccb3186-601f-467c-b51b-99639536e65a","Type":"ContainerStarted","Data":"8f5c600814399d8a3438712091f5309fc49ee3d0827f87b520bcb8e88017d60c"} Sep 08 07:08:19.814409 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:19.814365 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bd9f9987c-9c2q8" podStartSLOduration=1.81434995 podStartE2EDuration="1.81434995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:08:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:08:19.813132471 +0000 UTC m=+214.491095588" watchObservedRunningTime="2026-09-08 07:08:19.81434995 +0000 UTC m=+214.492313066" Sep 08 07:08:28.690124 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:28.690083 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:28.690603 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:28.690163 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:28.694813 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:28.694789 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:28.832137 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:28.832107 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:08:28.886265 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:28.886229 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d5b45f99d-9zr2z"] Sep 08 07:08:53.906882 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:53.906835 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d5b45f99d-9zr2z" podUID="ec0cac89-3da1-4c31-8713-06d09d19092b" containerName="console" containerID="cri-o://5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4" gracePeriod=15 Sep 08 07:08:54.144433 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.144409 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:08:54.280542 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.280493 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-console-config\") pod \"ec0cac89-3da1-4c31-8713-06d09d19092b\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " Sep 08 07:08:54.280542 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.280542 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-oauth-serving-cert\") pod \"ec0cac89-3da1-4c31-8713-06d09d19092b\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " Sep 08 07:08:54.280806 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.280595 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tpfq\" (UniqueName: \"kubernetes.io/projected/ec0cac89-3da1-4c31-8713-06d09d19092b-kube-api-access-6tpfq\") pod \"ec0cac89-3da1-4c31-8713-06d09d19092b\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " Sep 08 07:08:54.280806 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.280637 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-serving-cert\") pod \"ec0cac89-3da1-4c31-8713-06d09d19092b\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " Sep 08 07:08:54.280806 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.280655 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-service-ca\") pod \"ec0cac89-3da1-4c31-8713-06d09d19092b\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " Sep 08 07:08:54.280806 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.280766 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-oauth-config\") pod \"ec0cac89-3da1-4c31-8713-06d09d19092b\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " Sep 08 07:08:54.281016 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.280820 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-trusted-ca-bundle\") pod \"ec0cac89-3da1-4c31-8713-06d09d19092b\" (UID: \"ec0cac89-3da1-4c31-8713-06d09d19092b\") " Sep 08 07:08:54.281016 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.280996 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-console-config" (OuterVolumeSpecName: "console-config") pod "ec0cac89-3da1-4c31-8713-06d09d19092b" (UID: "ec0cac89-3da1-4c31-8713-06d09d19092b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:08:54.281117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.281096 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-console-config\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:08:54.281161 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.281091 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ec0cac89-3da1-4c31-8713-06d09d19092b" (UID: "ec0cac89-3da1-4c31-8713-06d09d19092b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:08:54.281161 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.281095 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-service-ca" (OuterVolumeSpecName: "service-ca") pod "ec0cac89-3da1-4c31-8713-06d09d19092b" (UID: "ec0cac89-3da1-4c31-8713-06d09d19092b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:08:54.281391 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.281357 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ec0cac89-3da1-4c31-8713-06d09d19092b" (UID: "ec0cac89-3da1-4c31-8713-06d09d19092b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:08:54.282904 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.282873 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ec0cac89-3da1-4c31-8713-06d09d19092b" (UID: "ec0cac89-3da1-4c31-8713-06d09d19092b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:08:54.283001 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.282906 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ec0cac89-3da1-4c31-8713-06d09d19092b" (UID: "ec0cac89-3da1-4c31-8713-06d09d19092b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:08:54.283001 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.282923 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec0cac89-3da1-4c31-8713-06d09d19092b-kube-api-access-6tpfq" (OuterVolumeSpecName: "kube-api-access-6tpfq") pod "ec0cac89-3da1-4c31-8713-06d09d19092b" (UID: "ec0cac89-3da1-4c31-8713-06d09d19092b"). InnerVolumeSpecName "kube-api-access-6tpfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:08:54.382123 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.382073 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-oauth-serving-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:08:54.382123 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.382117 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tpfq\" (UniqueName: \"kubernetes.io/projected/ec0cac89-3da1-4c31-8713-06d09d19092b-kube-api-access-6tpfq\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:08:54.382123 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.382128 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-serving-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:08:54.382123 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.382137 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:08:54.382405 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.382146 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec0cac89-3da1-4c31-8713-06d09d19092b-console-oauth-config\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:08:54.382405 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.382154 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec0cac89-3da1-4c31-8713-06d09d19092b-trusted-ca-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:08:54.924513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.924481 2571 generic.go:356] "Generic (PLEG): container finished" podID="ec0cac89-3da1-4c31-8713-06d09d19092b" containerID="5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4" exitCode=0 Sep 08 07:08:54.924918 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.924523 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5b45f99d-9zr2z" Sep 08 07:08:54.924918 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.924552 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5b45f99d-9zr2z" event={"ID":"ec0cac89-3da1-4c31-8713-06d09d19092b","Type":"ContainerDied","Data":"5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4"} Sep 08 07:08:54.924918 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.924588 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5b45f99d-9zr2z" event={"ID":"ec0cac89-3da1-4c31-8713-06d09d19092b","Type":"ContainerDied","Data":"de677007c1705a2a8141e51a3053a98d7941b78cbbac1c6f0cf55f1c54bdbcb2"} Sep 08 07:08:54.924918 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.924607 2571 scope.go:117] "RemoveContainer" containerID="5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4" Sep 08 07:08:54.933417 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.933402 2571 scope.go:117] "RemoveContainer" containerID="5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4" Sep 08 07:08:54.933666 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:08:54.933646 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4\": container with ID starting with 5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4 not found: ID does not exist" containerID="5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4" Sep 08 07:08:54.933713 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.933673 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4"} err="failed to get container status \"5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4\": rpc error: code = NotFound desc = could not find container \"5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4\": container with ID starting with 5111d7c677605fe97821f03d0fbfc88bd4ea63ec2bf2765f6e67d93d9d8f57b4 not found: ID does not exist" Sep 08 07:08:54.944730 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.944706 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d5b45f99d-9zr2z"] Sep 08 07:08:54.948683 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:54.948665 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d5b45f99d-9zr2z"] Sep 08 07:08:55.807477 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:08:55.807445 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec0cac89-3da1-4c31-8713-06d09d19092b" path="/var/lib/kubelet/pods/ec0cac89-3da1-4c31-8713-06d09d19092b/volumes" Sep 08 07:09:19.568839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.568762 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn"] Sep 08 07:09:19.569272 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.569063 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec0cac89-3da1-4c31-8713-06d09d19092b" containerName="console" Sep 08 07:09:19.569272 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.569074 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec0cac89-3da1-4c31-8713-06d09d19092b" containerName="console" Sep 08 07:09:19.569272 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.569119 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec0cac89-3da1-4c31-8713-06d09d19092b" containerName="console" Sep 08 07:09:19.571982 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.571965 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:19.574643 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.574623 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 07:09:19.574747 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.574663 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6b6kz\"" Sep 08 07:09:19.575735 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.575717 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 07:09:19.580920 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.580894 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn"] Sep 08 07:09:19.666337 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.666297 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk6kf\" (UniqueName: \"kubernetes.io/projected/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-kube-api-access-jk6kf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:19.666509 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.666357 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:19.666509 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.666419 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:19.767640 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.767589 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk6kf\" (UniqueName: \"kubernetes.io/projected/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-kube-api-access-jk6kf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:19.767752 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.767653 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:19.767752 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.767696 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:19.768011 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.767996 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:19.768072 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.768055 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:19.776920 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.776893 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk6kf\" (UniqueName: \"kubernetes.io/projected/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-kube-api-access-jk6kf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:19.880727 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:19.880645 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:20.150783 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:20.150753 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn"] Sep 08 07:09:20.152860 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:09:20.152835 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5fe0a1_57ec_4842_a3a9_f02e8c7a8c2a.slice/crio-426a6f6cb7a5811b90092478134554111a81638f7aa52db12420ab7a4507dc0f WatchSource:0}: Error finding container 426a6f6cb7a5811b90092478134554111a81638f7aa52db12420ab7a4507dc0f: Status 404 returned error can't find the container with id 426a6f6cb7a5811b90092478134554111a81638f7aa52db12420ab7a4507dc0f Sep 08 07:09:21.021697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:21.021657 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" event={"ID":"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a","Type":"ContainerStarted","Data":"426a6f6cb7a5811b90092478134554111a81638f7aa52db12420ab7a4507dc0f"} Sep 08 07:09:27.050773 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:27.050736 2571 generic.go:356] "Generic (PLEG): container finished" podID="ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" containerID="d05b108b42ae58aac16b4eaa4ba4d71e61b287d525fb01e2790d6c206c83e139" exitCode=0 Sep 08 07:09:27.051294 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:27.050790 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" event={"ID":"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a","Type":"ContainerDied","Data":"d05b108b42ae58aac16b4eaa4ba4d71e61b287d525fb01e2790d6c206c83e139"} Sep 08 07:09:30.063683 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:30.063645 2571 generic.go:356] "Generic (PLEG): container finished" podID="ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" containerID="94f6c45401dcb0e4ae3ef3355a6aa351556df400b2a456a7905db96b42123219" exitCode=0 Sep 08 07:09:30.064159 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:30.063685 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" event={"ID":"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a","Type":"ContainerDied","Data":"94f6c45401dcb0e4ae3ef3355a6aa351556df400b2a456a7905db96b42123219"} Sep 08 07:09:37.093931 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:37.093895 2571 generic.go:356] "Generic (PLEG): container finished" podID="ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" containerID="bd35342429933dabf0d6793d4857dc7d596c16f2bae3a62232114e908d96dac1" exitCode=0 Sep 08 07:09:37.094313 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:37.093951 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" event={"ID":"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a","Type":"ContainerDied","Data":"bd35342429933dabf0d6793d4857dc7d596c16f2bae3a62232114e908d96dac1"} Sep 08 07:09:38.219838 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:38.219815 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:38.330238 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:38.330181 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk6kf\" (UniqueName: \"kubernetes.io/projected/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-kube-api-access-jk6kf\") pod \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " Sep 08 07:09:38.330422 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:38.330257 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-util\") pod \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " Sep 08 07:09:38.330422 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:38.330286 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-bundle\") pod \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\" (UID: \"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a\") " Sep 08 07:09:38.331079 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:38.331052 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-bundle" (OuterVolumeSpecName: "bundle") pod "ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" (UID: "ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:09:38.332340 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:38.332315 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-kube-api-access-jk6kf" (OuterVolumeSpecName: "kube-api-access-jk6kf") pod "ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" (UID: "ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a"). InnerVolumeSpecName "kube-api-access-jk6kf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:09:38.334809 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:38.334784 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-util" (OuterVolumeSpecName: "util") pod "ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" (UID: "ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:09:38.431416 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:38.431332 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk6kf\" (UniqueName: \"kubernetes.io/projected/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-kube-api-access-jk6kf\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:09:38.431416 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:38.431362 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-util\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:09:38.431416 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:38.431372 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:09:39.102936 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:39.102905 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" Sep 08 07:09:39.102936 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:39.102919 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lw5dn" event={"ID":"ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a","Type":"ContainerDied","Data":"426a6f6cb7a5811b90092478134554111a81638f7aa52db12420ab7a4507dc0f"} Sep 08 07:09:39.103152 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:39.102997 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="426a6f6cb7a5811b90092478134554111a81638f7aa52db12420ab7a4507dc0f" Sep 08 07:09:42.462644 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.462613 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq"] Sep 08 07:09:42.463003 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.462891 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" containerName="util" Sep 08 07:09:42.463003 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.462902 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" containerName="util" Sep 08 07:09:42.463003 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.462910 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" containerName="pull" Sep 08 07:09:42.463003 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.462916 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" containerName="pull" Sep 08 07:09:42.463003 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.462924 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" containerName="extract" Sep 08 07:09:42.463003 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.462930 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" containerName="extract" Sep 08 07:09:42.463003 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.462974 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff5fe0a1-57ec-4842-a3a9-f02e8c7a8c2a" containerName="extract" Sep 08 07:09:42.466248 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.466231 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" Sep 08 07:09:42.468471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.468446 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 08 07:09:42.468617 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.468520 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 08 07:09:42.468617 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.468586 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5s9vs\"" Sep 08 07:09:42.483918 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.483893 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq"] Sep 08 07:09:42.562182 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.562139 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/634cc47b-51e5-43bd-88e1-445c539e06f5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2rlpq\" (UID: \"634cc47b-51e5-43bd-88e1-445c539e06f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" Sep 08 07:09:42.562375 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.562200 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5ndm\" (UniqueName: \"kubernetes.io/projected/634cc47b-51e5-43bd-88e1-445c539e06f5-kube-api-access-p5ndm\") pod \"cert-manager-operator-controller-manager-55c444bf86-2rlpq\" (UID: \"634cc47b-51e5-43bd-88e1-445c539e06f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" Sep 08 07:09:42.663446 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.663387 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/634cc47b-51e5-43bd-88e1-445c539e06f5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2rlpq\" (UID: \"634cc47b-51e5-43bd-88e1-445c539e06f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" Sep 08 07:09:42.663618 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.663464 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5ndm\" (UniqueName: \"kubernetes.io/projected/634cc47b-51e5-43bd-88e1-445c539e06f5-kube-api-access-p5ndm\") pod \"cert-manager-operator-controller-manager-55c444bf86-2rlpq\" (UID: \"634cc47b-51e5-43bd-88e1-445c539e06f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" Sep 08 07:09:42.663807 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.663786 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/634cc47b-51e5-43bd-88e1-445c539e06f5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2rlpq\" (UID: \"634cc47b-51e5-43bd-88e1-445c539e06f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" Sep 08 07:09:42.671611 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.671578 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5ndm\" (UniqueName: \"kubernetes.io/projected/634cc47b-51e5-43bd-88e1-445c539e06f5-kube-api-access-p5ndm\") pod \"cert-manager-operator-controller-manager-55c444bf86-2rlpq\" (UID: \"634cc47b-51e5-43bd-88e1-445c539e06f5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" Sep 08 07:09:42.774781 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:42.774701 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" Sep 08 07:09:43.050101 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:43.050077 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq"] Sep 08 07:09:43.052852 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:09:43.052819 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634cc47b_51e5_43bd_88e1_445c539e06f5.slice/crio-1da3f4f32fea7c512505beeb49d9b7408b132fbe7b3dd420419bb8feda100f88 WatchSource:0}: Error finding container 1da3f4f32fea7c512505beeb49d9b7408b132fbe7b3dd420419bb8feda100f88: Status 404 returned error can't find the container with id 1da3f4f32fea7c512505beeb49d9b7408b132fbe7b3dd420419bb8feda100f88 Sep 08 07:09:43.120699 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:43.120667 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" event={"ID":"634cc47b-51e5-43bd-88e1-445c539e06f5","Type":"ContainerStarted","Data":"1da3f4f32fea7c512505beeb49d9b7408b132fbe7b3dd420419bb8feda100f88"} Sep 08 07:09:45.134939 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:45.134849 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" event={"ID":"634cc47b-51e5-43bd-88e1-445c539e06f5","Type":"ContainerStarted","Data":"10b38faada452382677f923ac6bd1c4e95ba2664bf44f68b0d741c0bb4138fee"} Sep 08 07:09:45.157136 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:45.157078 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2rlpq" podStartSLOduration=1.435237913 podStartE2EDuration="3.15705718s" totalImagesPullingTime="1.721819267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:09:45.154555971 +0000 UTC m=+299.832519112" watchObservedRunningTime="2026-09-08 07:09:45.15705718 +0000 UTC m=+299.835020297" Sep 08 07:09:45.692966 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:45.692942 2571 kubelet.go:1648] "Image garbage collection succeeded" Sep 08 07:09:46.189457 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.189421 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l"] Sep 08 07:09:46.192814 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.192795 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.195435 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.195408 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6b6kz\"" Sep 08 07:09:46.195581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.195460 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 07:09:46.195581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.195503 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 07:09:46.203966 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.203941 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l"] Sep 08 07:09:46.298945 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.298908 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.299102 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.298948 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zspr\" (UniqueName: \"kubernetes.io/projected/d2e71074-3ad7-4f96-8fc6-19698baaa804-kube-api-access-7zspr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.299102 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.299068 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.399929 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.399891 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.400092 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.399958 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.400092 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.399977 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zspr\" (UniqueName: \"kubernetes.io/projected/d2e71074-3ad7-4f96-8fc6-19698baaa804-kube-api-access-7zspr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.400412 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.400390 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.400456 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.400390 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.411952 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.411913 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zspr\" (UniqueName: \"kubernetes.io/projected/d2e71074-3ad7-4f96-8fc6-19698baaa804-kube-api-access-7zspr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.501800 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.501765 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:09:46.771317 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:46.771290 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l"] Sep 08 07:09:46.773552 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:09:46.773523 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e71074_3ad7_4f96_8fc6_19698baaa804.slice/crio-5e04da303ed85bd109fb8350b64c690149eb88970741e6734ce2d0cbd4e7f76a WatchSource:0}: Error finding container 5e04da303ed85bd109fb8350b64c690149eb88970741e6734ce2d0cbd4e7f76a: Status 404 returned error can't find the container with id 5e04da303ed85bd109fb8350b64c690149eb88970741e6734ce2d0cbd4e7f76a Sep 08 07:09:47.143172 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:47.143084 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2e71074-3ad7-4f96-8fc6-19698baaa804" containerID="806994e5ddd10028f24ba16ab19f160c005842d8c9d1d3fe2b8e15ccf7dbe7ad" exitCode=0 Sep 08 07:09:47.143327 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:47.143166 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" event={"ID":"d2e71074-3ad7-4f96-8fc6-19698baaa804","Type":"ContainerDied","Data":"806994e5ddd10028f24ba16ab19f160c005842d8c9d1d3fe2b8e15ccf7dbe7ad"} Sep 08 07:09:47.143327 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:47.143225 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" event={"ID":"d2e71074-3ad7-4f96-8fc6-19698baaa804","Type":"ContainerStarted","Data":"5e04da303ed85bd109fb8350b64c690149eb88970741e6734ce2d0cbd4e7f76a"} Sep 08 07:09:47.144101 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:47.144087 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 07:09:48.764917 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:48.764882 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh"] Sep 08 07:09:48.768198 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:48.768181 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" Sep 08 07:09:48.770632 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:48.770609 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 08 07:09:48.770632 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:48.770611 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-vxf2g\"" Sep 08 07:09:48.771612 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:48.771595 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 08 07:09:48.777392 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:48.777371 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh"] Sep 08 07:09:48.924787 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:48.924755 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1783620-9ed5-4201-a884-e254595eaa3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9g8gh\" (UID: \"b1783620-9ed5-4201-a884-e254595eaa3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" Sep 08 07:09:48.924967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:48.924820 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njxsz\" (UniqueName: \"kubernetes.io/projected/b1783620-9ed5-4201-a884-e254595eaa3d-kube-api-access-njxsz\") pod \"cert-manager-cainjector-559cdd8c87-9g8gh\" (UID: \"b1783620-9ed5-4201-a884-e254595eaa3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" Sep 08 07:09:49.025666 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:49.025575 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njxsz\" (UniqueName: \"kubernetes.io/projected/b1783620-9ed5-4201-a884-e254595eaa3d-kube-api-access-njxsz\") pod \"cert-manager-cainjector-559cdd8c87-9g8gh\" (UID: \"b1783620-9ed5-4201-a884-e254595eaa3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" Sep 08 07:09:49.025666 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:49.025633 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1783620-9ed5-4201-a884-e254595eaa3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9g8gh\" (UID: \"b1783620-9ed5-4201-a884-e254595eaa3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" Sep 08 07:09:49.034708 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:49.034672 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1783620-9ed5-4201-a884-e254595eaa3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9g8gh\" (UID: \"b1783620-9ed5-4201-a884-e254595eaa3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" Sep 08 07:09:49.035013 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:49.034994 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njxsz\" (UniqueName: \"kubernetes.io/projected/b1783620-9ed5-4201-a884-e254595eaa3d-kube-api-access-njxsz\") pod \"cert-manager-cainjector-559cdd8c87-9g8gh\" (UID: \"b1783620-9ed5-4201-a884-e254595eaa3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" Sep 08 07:09:49.085019 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:49.084978 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" Sep 08 07:09:49.353602 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:49.353576 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh"] Sep 08 07:09:49.356003 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:09:49.355970 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1783620_9ed5_4201_a884_e254595eaa3d.slice/crio-8f594af74a57d53530f7932ea23ab6ca727521d35e8269e83a9708c9237e0137 WatchSource:0}: Error finding container 8f594af74a57d53530f7932ea23ab6ca727521d35e8269e83a9708c9237e0137: Status 404 returned error can't find the container with id 8f594af74a57d53530f7932ea23ab6ca727521d35e8269e83a9708c9237e0137 Sep 08 07:09:50.155671 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:09:50.155641 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" event={"ID":"b1783620-9ed5-4201-a884-e254595eaa3d","Type":"ContainerStarted","Data":"8f594af74a57d53530f7932ea23ab6ca727521d35e8269e83a9708c9237e0137"} Sep 08 07:10:06.226228 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:06.226179 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2e71074-3ad7-4f96-8fc6-19698baaa804" containerID="ef9a804ff6f58615f395e6c93a94ee64b4e093c1d3bf47cb301f8e32f4aafd53" exitCode=0 Sep 08 07:10:06.226748 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:06.226252 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" event={"ID":"d2e71074-3ad7-4f96-8fc6-19698baaa804","Type":"ContainerDied","Data":"ef9a804ff6f58615f395e6c93a94ee64b4e093c1d3bf47cb301f8e32f4aafd53"} Sep 08 07:10:06.995198 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:06.995160 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mmkp6"] Sep 08 07:10:06.998434 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:06.998413 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mmkp6" Sep 08 07:10:07.000664 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.000640 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-4ct84\"" Sep 08 07:10:07.008890 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.008866 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mmkp6"] Sep 08 07:10:07.066524 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.066482 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/221ef323-d9a1-4aee-abd5-e42bd730146b-bound-sa-token\") pod \"cert-manager-55b4d954d5-mmkp6\" (UID: \"221ef323-d9a1-4aee-abd5-e42bd730146b\") " pod="cert-manager/cert-manager-55b4d954d5-mmkp6" Sep 08 07:10:07.066687 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.066591 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rszm2\" (UniqueName: \"kubernetes.io/projected/221ef323-d9a1-4aee-abd5-e42bd730146b-kube-api-access-rszm2\") pod \"cert-manager-55b4d954d5-mmkp6\" (UID: \"221ef323-d9a1-4aee-abd5-e42bd730146b\") " pod="cert-manager/cert-manager-55b4d954d5-mmkp6" Sep 08 07:10:07.167612 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.167575 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/221ef323-d9a1-4aee-abd5-e42bd730146b-bound-sa-token\") pod \"cert-manager-55b4d954d5-mmkp6\" (UID: \"221ef323-d9a1-4aee-abd5-e42bd730146b\") " pod="cert-manager/cert-manager-55b4d954d5-mmkp6" Sep 08 07:10:07.167778 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.167652 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rszm2\" (UniqueName: \"kubernetes.io/projected/221ef323-d9a1-4aee-abd5-e42bd730146b-kube-api-access-rszm2\") pod \"cert-manager-55b4d954d5-mmkp6\" (UID: \"221ef323-d9a1-4aee-abd5-e42bd730146b\") " pod="cert-manager/cert-manager-55b4d954d5-mmkp6" Sep 08 07:10:07.175661 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.175627 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/221ef323-d9a1-4aee-abd5-e42bd730146b-bound-sa-token\") pod \"cert-manager-55b4d954d5-mmkp6\" (UID: \"221ef323-d9a1-4aee-abd5-e42bd730146b\") " pod="cert-manager/cert-manager-55b4d954d5-mmkp6" Sep 08 07:10:07.175833 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.175814 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rszm2\" (UniqueName: \"kubernetes.io/projected/221ef323-d9a1-4aee-abd5-e42bd730146b-kube-api-access-rszm2\") pod \"cert-manager-55b4d954d5-mmkp6\" (UID: \"221ef323-d9a1-4aee-abd5-e42bd730146b\") " pod="cert-manager/cert-manager-55b4d954d5-mmkp6" Sep 08 07:10:07.232298 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.232265 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2e71074-3ad7-4f96-8fc6-19698baaa804" containerID="5fe4046c212d0a4c6be9401a59eb5e840a0e9b5a6a3d44a32e838be9c6af4900" exitCode=0 Sep 08 07:10:07.232665 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.232354 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" event={"ID":"d2e71074-3ad7-4f96-8fc6-19698baaa804","Type":"ContainerDied","Data":"5fe4046c212d0a4c6be9401a59eb5e840a0e9b5a6a3d44a32e838be9c6af4900"} Sep 08 07:10:07.308488 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.308409 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mmkp6" Sep 08 07:10:07.575628 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:07.575540 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mmkp6"] Sep 08 07:10:07.578334 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:10:07.578292 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod221ef323_d9a1_4aee_abd5_e42bd730146b.slice/crio-8c8f444d896783bba6512bf3cc8587375f484fb92050af1c5a28303d51065926 WatchSource:0}: Error finding container 8c8f444d896783bba6512bf3cc8587375f484fb92050af1c5a28303d51065926: Status 404 returned error can't find the container with id 8c8f444d896783bba6512bf3cc8587375f484fb92050af1c5a28303d51065926 Sep 08 07:10:08.237532 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.237497 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mmkp6" event={"ID":"221ef323-d9a1-4aee-abd5-e42bd730146b","Type":"ContainerStarted","Data":"8c8f444d896783bba6512bf3cc8587375f484fb92050af1c5a28303d51065926"} Sep 08 07:10:08.365974 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.365947 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:10:08.376793 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.376765 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-util\") pod \"d2e71074-3ad7-4f96-8fc6-19698baaa804\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " Sep 08 07:10:08.376922 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.376831 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zspr\" (UniqueName: \"kubernetes.io/projected/d2e71074-3ad7-4f96-8fc6-19698baaa804-kube-api-access-7zspr\") pod \"d2e71074-3ad7-4f96-8fc6-19698baaa804\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " Sep 08 07:10:08.376922 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.376864 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-bundle\") pod \"d2e71074-3ad7-4f96-8fc6-19698baaa804\" (UID: \"d2e71074-3ad7-4f96-8fc6-19698baaa804\") " Sep 08 07:10:08.377310 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.377282 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-bundle" (OuterVolumeSpecName: "bundle") pod "d2e71074-3ad7-4f96-8fc6-19698baaa804" (UID: "d2e71074-3ad7-4f96-8fc6-19698baaa804"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:10:08.378862 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.378833 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2e71074-3ad7-4f96-8fc6-19698baaa804-kube-api-access-7zspr" (OuterVolumeSpecName: "kube-api-access-7zspr") pod "d2e71074-3ad7-4f96-8fc6-19698baaa804" (UID: "d2e71074-3ad7-4f96-8fc6-19698baaa804"). InnerVolumeSpecName "kube-api-access-7zspr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:10:08.381744 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.381719 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-util" (OuterVolumeSpecName: "util") pod "d2e71074-3ad7-4f96-8fc6-19698baaa804" (UID: "d2e71074-3ad7-4f96-8fc6-19698baaa804"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:10:08.477419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.477378 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zspr\" (UniqueName: \"kubernetes.io/projected/d2e71074-3ad7-4f96-8fc6-19698baaa804-kube-api-access-7zspr\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:10:08.477419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.477414 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:10:08.477419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:08.477427 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2e71074-3ad7-4f96-8fc6-19698baaa804-util\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:10:09.244648 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:09.244612 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" event={"ID":"d2e71074-3ad7-4f96-8fc6-19698baaa804","Type":"ContainerDied","Data":"5e04da303ed85bd109fb8350b64c690149eb88970741e6734ce2d0cbd4e7f76a"} Sep 08 07:10:09.244648 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:09.244638 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f67k4l" Sep 08 07:10:09.245208 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:09.244662 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e04da303ed85bd109fb8350b64c690149eb88970741e6734ce2d0cbd4e7f76a" Sep 08 07:10:11.255474 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:11.255436 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mmkp6" event={"ID":"221ef323-d9a1-4aee-abd5-e42bd730146b","Type":"ContainerStarted","Data":"b94fb194cbc8242af1372c33c252412b5521a9ffde3d624e76ade0792b004021"} Sep 08 07:10:11.277679 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:11.277628 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-mmkp6" podStartSLOduration=2.072641948 podStartE2EDuration="5.277613611s" totalImagesPullingTime="3.204971663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:10:11.276649922 +0000 UTC m=+325.954613057" watchObservedRunningTime="2026-09-08 07:10:11.277613611 +0000 UTC m=+325.955576726" Sep 08 07:10:14.270581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:14.270540 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" event={"ID":"b1783620-9ed5-4201-a884-e254595eaa3d","Type":"ContainerStarted","Data":"9577c529d2fbf99e7d6ef5dfee00ee26beffcd68b883d7a26f8100e00ca03b30"} Sep 08 07:10:14.288861 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:14.288801 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g8gh" podStartSLOduration=1.605452755 podStartE2EDuration="26.288783602s" totalImagesPullingTime="24.683330847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:10:14.287181468 +0000 UTC m=+328.965144587" watchObservedRunningTime="2026-09-08 07:10:14.288783602 +0000 UTC m=+328.966746718" Sep 08 07:10:27.507846 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.507810 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp"] Sep 08 07:10:27.508243 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.508083 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e71074-3ad7-4f96-8fc6-19698baaa804" containerName="util" Sep 08 07:10:27.508243 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.508094 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e71074-3ad7-4f96-8fc6-19698baaa804" containerName="util" Sep 08 07:10:27.508243 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.508117 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e71074-3ad7-4f96-8fc6-19698baaa804" containerName="extract" Sep 08 07:10:27.508243 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.508123 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e71074-3ad7-4f96-8fc6-19698baaa804" containerName="extract" Sep 08 07:10:27.508243 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.508130 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e71074-3ad7-4f96-8fc6-19698baaa804" containerName="pull" Sep 08 07:10:27.508243 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.508134 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e71074-3ad7-4f96-8fc6-19698baaa804" containerName="pull" Sep 08 07:10:27.508243 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.508181 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e71074-3ad7-4f96-8fc6-19698baaa804" containerName="extract" Sep 08 07:10:27.512960 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.512942 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.517107 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.517080 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 08 07:10:27.517308 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.517087 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 08 07:10:27.517308 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.517087 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 08 07:10:27.517308 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.517188 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 08 07:10:27.517308 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.517208 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 08 07:10:27.517308 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.517230 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wvhpm\"" Sep 08 07:10:27.520366 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.520338 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp"] Sep 08 07:10:27.590048 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.590014 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t"] Sep 08 07:10:27.593759 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.593736 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:27.597787 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.597767 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 07:10:27.597914 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.597792 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6b6kz\"" Sep 08 07:10:27.598032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.598016 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 07:10:27.605023 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.604998 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t"] Sep 08 07:10:27.636123 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.636090 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-manager-config\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.636303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.636132 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-metrics-cert\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.636303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.636157 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wzfm\" (UniqueName: \"kubernetes.io/projected/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-kube-api-access-2wzfm\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.636303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.636245 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-cert\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.736676 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.736636 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:27.736676 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.736676 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-manager-config\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.736887 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.736708 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-metrics-cert\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.736887 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.736752 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:27.736887 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.736846 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wzfm\" (UniqueName: \"kubernetes.io/projected/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-kube-api-access-2wzfm\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.737000 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.736890 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-cert\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.737000 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.736929 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9pkw\" (UniqueName: \"kubernetes.io/projected/2e394465-19c7-4cbe-9f88-9788d82305fd-kube-api-access-q9pkw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:27.737378 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.737353 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-manager-config\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.739268 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.739246 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-metrics-cert\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.739375 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.739293 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-cert\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.745753 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.745733 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wzfm\" (UniqueName: \"kubernetes.io/projected/ed7c8ccd-be8f-4ac2-890d-d63fab8388bc-kube-api-access-2wzfm\") pod \"lws-controller-manager-767bfdf788-c2kfp\" (UID: \"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc\") " pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.823706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.823614 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:27.837298 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.837274 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9pkw\" (UniqueName: \"kubernetes.io/projected/2e394465-19c7-4cbe-9f88-9788d82305fd-kube-api-access-q9pkw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:27.837383 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.837328 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:27.837444 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.837380 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:27.837674 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.837650 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:27.837738 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.837709 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:27.846320 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.846294 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9pkw\" (UniqueName: \"kubernetes.io/projected/2e394465-19c7-4cbe-9f88-9788d82305fd-kube-api-access-q9pkw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:27.903337 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:27.903306 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:28.096130 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:28.096073 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp"] Sep 08 07:10:28.098599 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:10:28.098574 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7c8ccd_be8f_4ac2_890d_d63fab8388bc.slice/crio-05a60fdfa2e4f7b65b761f756c3764fe88ebe99cc90ecd5ef189389bff7215b6 WatchSource:0}: Error finding container 05a60fdfa2e4f7b65b761f756c3764fe88ebe99cc90ecd5ef189389bff7215b6: Status 404 returned error can't find the container with id 05a60fdfa2e4f7b65b761f756c3764fe88ebe99cc90ecd5ef189389bff7215b6 Sep 08 07:10:28.176619 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:28.176592 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t"] Sep 08 07:10:28.178419 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:10:28.178393 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e394465_19c7_4cbe_9f88_9788d82305fd.slice/crio-ac518965f18a01e4e846f9a0c6004f6c4bf9e509150aad95c9e1db578cd7b6d0 WatchSource:0}: Error finding container ac518965f18a01e4e846f9a0c6004f6c4bf9e509150aad95c9e1db578cd7b6d0: Status 404 returned error can't find the container with id ac518965f18a01e4e846f9a0c6004f6c4bf9e509150aad95c9e1db578cd7b6d0 Sep 08 07:10:28.334973 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:28.334934 2571 generic.go:356] "Generic (PLEG): container finished" podID="2e394465-19c7-4cbe-9f88-9788d82305fd" containerID="8a9d978d2de8594113c694fb4f0d73a5c471eb1905991512b59e1733e1dc7544" exitCode=0 Sep 08 07:10:28.335144 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:28.335022 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" event={"ID":"2e394465-19c7-4cbe-9f88-9788d82305fd","Type":"ContainerDied","Data":"8a9d978d2de8594113c694fb4f0d73a5c471eb1905991512b59e1733e1dc7544"} Sep 08 07:10:28.335144 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:28.335079 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" event={"ID":"2e394465-19c7-4cbe-9f88-9788d82305fd","Type":"ContainerStarted","Data":"ac518965f18a01e4e846f9a0c6004f6c4bf9e509150aad95c9e1db578cd7b6d0"} Sep 08 07:10:28.336321 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:28.336298 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" event={"ID":"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc","Type":"ContainerStarted","Data":"05a60fdfa2e4f7b65b761f756c3764fe88ebe99cc90ecd5ef189389bff7215b6"} Sep 08 07:10:30.349833 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:30.349798 2571 generic.go:356] "Generic (PLEG): container finished" podID="2e394465-19c7-4cbe-9f88-9788d82305fd" containerID="0cbfe1c19b94cb86c17e64627e10bdc89febdd1d39906ca8993e9875a97d5279" exitCode=0 Sep 08 07:10:30.350273 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:30.349883 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" event={"ID":"2e394465-19c7-4cbe-9f88-9788d82305fd","Type":"ContainerDied","Data":"0cbfe1c19b94cb86c17e64627e10bdc89febdd1d39906ca8993e9875a97d5279"} Sep 08 07:10:30.351496 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:30.351476 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" event={"ID":"ed7c8ccd-be8f-4ac2-890d-d63fab8388bc","Type":"ContainerStarted","Data":"d8ef759f950f239761dcb348da6fa7a236214ecdb1b693e5293f953ee5681e6f"} Sep 08 07:10:30.351618 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:30.351605 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:30.382985 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:30.382941 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" podStartSLOduration=1.76816069 podStartE2EDuration="3.382925742s" totalImagesPullingTime="1.614765052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:10:30.381357736 +0000 UTC m=+345.059320878" watchObservedRunningTime="2026-09-08 07:10:30.382925742 +0000 UTC m=+345.060888891" Sep 08 07:10:31.358538 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:31.358504 2571 generic.go:356] "Generic (PLEG): container finished" podID="2e394465-19c7-4cbe-9f88-9788d82305fd" containerID="33184b57edd7d5484764955e50e23f05b853ab7136131253a8e11917f050f5e4" exitCode=0 Sep 08 07:10:31.358900 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:31.358601 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" event={"ID":"2e394465-19c7-4cbe-9f88-9788d82305fd","Type":"ContainerDied","Data":"33184b57edd7d5484764955e50e23f05b853ab7136131253a8e11917f050f5e4"} Sep 08 07:10:32.492395 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:32.492367 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:32.576404 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:32.576368 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-util\") pod \"2e394465-19c7-4cbe-9f88-9788d82305fd\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " Sep 08 07:10:32.576404 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:32.576404 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-bundle\") pod \"2e394465-19c7-4cbe-9f88-9788d82305fd\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " Sep 08 07:10:32.576625 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:32.576471 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9pkw\" (UniqueName: \"kubernetes.io/projected/2e394465-19c7-4cbe-9f88-9788d82305fd-kube-api-access-q9pkw\") pod \"2e394465-19c7-4cbe-9f88-9788d82305fd\" (UID: \"2e394465-19c7-4cbe-9f88-9788d82305fd\") " Sep 08 07:10:32.577201 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:32.577174 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-bundle" (OuterVolumeSpecName: "bundle") pod "2e394465-19c7-4cbe-9f88-9788d82305fd" (UID: "2e394465-19c7-4cbe-9f88-9788d82305fd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:10:32.578551 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:32.578528 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e394465-19c7-4cbe-9f88-9788d82305fd-kube-api-access-q9pkw" (OuterVolumeSpecName: "kube-api-access-q9pkw") pod "2e394465-19c7-4cbe-9f88-9788d82305fd" (UID: "2e394465-19c7-4cbe-9f88-9788d82305fd"). InnerVolumeSpecName "kube-api-access-q9pkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:10:32.581635 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:32.581613 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-util" (OuterVolumeSpecName: "util") pod "2e394465-19c7-4cbe-9f88-9788d82305fd" (UID: "2e394465-19c7-4cbe-9f88-9788d82305fd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:10:32.677822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:32.677723 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9pkw\" (UniqueName: \"kubernetes.io/projected/2e394465-19c7-4cbe-9f88-9788d82305fd-kube-api-access-q9pkw\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:10:32.677822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:32.677767 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-util\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:10:32.677822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:32.677777 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e394465-19c7-4cbe-9f88-9788d82305fd-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:10:33.369597 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:33.369567 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" event={"ID":"2e394465-19c7-4cbe-9f88-9788d82305fd","Type":"ContainerDied","Data":"ac518965f18a01e4e846f9a0c6004f6c4bf9e509150aad95c9e1db578cd7b6d0"} Sep 08 07:10:33.369767 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:33.369614 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac518965f18a01e4e846f9a0c6004f6c4bf9e509150aad95c9e1db578cd7b6d0" Sep 08 07:10:33.369767 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:33.369626 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afq9f2t" Sep 08 07:10:41.361506 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:41.361468 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-767bfdf788-c2kfp" Sep 08 07:10:41.476188 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:41.476154 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bd9f9987c-9c2q8"] Sep 08 07:10:44.797004 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.796968 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8"] Sep 08 07:10:44.797469 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.797343 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e394465-19c7-4cbe-9f88-9788d82305fd" containerName="pull" Sep 08 07:10:44.797469 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.797358 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e394465-19c7-4cbe-9f88-9788d82305fd" containerName="pull" Sep 08 07:10:44.797469 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.797369 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e394465-19c7-4cbe-9f88-9788d82305fd" containerName="util" Sep 08 07:10:44.797469 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.797374 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e394465-19c7-4cbe-9f88-9788d82305fd" containerName="util" Sep 08 07:10:44.797469 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.797387 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e394465-19c7-4cbe-9f88-9788d82305fd" containerName="extract" Sep 08 07:10:44.797469 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.797393 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e394465-19c7-4cbe-9f88-9788d82305fd" containerName="extract" Sep 08 07:10:44.797469 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.797448 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e394465-19c7-4cbe-9f88-9788d82305fd" containerName="extract" Sep 08 07:10:44.840850 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.840817 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8"] Sep 08 07:10:44.840997 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.840921 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:44.844895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.844871 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 08 07:10:44.845073 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.845054 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nw77s\"" Sep 08 07:10:44.845149 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.845059 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 08 07:10:44.845338 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.845324 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 08 07:10:44.845724 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.845710 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 08 07:10:44.916194 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.916156 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6"] Sep 08 07:10:44.918686 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.918666 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:44.921606 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.921587 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 07:10:44.921890 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.921876 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 07:10:44.921967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.921881 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6b6kz\"" Sep 08 07:10:44.928482 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.928456 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6"] Sep 08 07:10:44.983279 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.983242 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsk49\" (UniqueName: \"kubernetes.io/projected/8a2524d1-876e-4db1-ba27-84151cdfc850-kube-api-access-zsk49\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:44.983498 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.983312 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a2524d1-876e-4db1-ba27-84151cdfc850-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:44.983498 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.983338 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a2524d1-876e-4db1-ba27-84151cdfc850-manifests\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:44.983498 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.983385 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a2524d1-876e-4db1-ba27-84151cdfc850-tmp\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:44.983498 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:44.983454 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a2524d1-876e-4db1-ba27-84151cdfc850-webhook-cert\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.084678 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.084590 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a2524d1-876e-4db1-ba27-84151cdfc850-webhook-cert\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.084678 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.084631 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsk49\" (UniqueName: \"kubernetes.io/projected/8a2524d1-876e-4db1-ba27-84151cdfc850-kube-api-access-zsk49\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.084862 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.084761 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a2524d1-876e-4db1-ba27-84151cdfc850-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.084862 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.084792 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a2524d1-876e-4db1-ba27-84151cdfc850-manifests\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.084862 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.084829 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxd5h\" (UniqueName: \"kubernetes.io/projected/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-kube-api-access-lxd5h\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:45.084976 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.084950 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:45.085042 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.085025 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a2524d1-876e-4db1-ba27-84151cdfc850-tmp\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.085083 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.085063 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:45.085236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.085198 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a2524d1-876e-4db1-ba27-84151cdfc850-manifests\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.087080 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.087056 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a2524d1-876e-4db1-ba27-84151cdfc850-tmp\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.087354 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.087338 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a2524d1-876e-4db1-ba27-84151cdfc850-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.087484 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.087462 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a2524d1-876e-4db1-ba27-84151cdfc850-webhook-cert\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.093232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.093194 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsk49\" (UniqueName: \"kubernetes.io/projected/8a2524d1-876e-4db1-ba27-84151cdfc850-kube-api-access-zsk49\") pod \"opendatahub-operator-controller-manager-79c9ffb96f-2bss8\" (UID: \"8a2524d1-876e-4db1-ba27-84151cdfc850\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.150367 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.150331 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:45.185570 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.185526 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxd5h\" (UniqueName: \"kubernetes.io/projected/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-kube-api-access-lxd5h\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:45.185785 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.185584 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:45.185785 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.185634 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:45.186078 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.186055 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:45.186144 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.186113 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:45.195323 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.195260 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxd5h\" (UniqueName: \"kubernetes.io/projected/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-kube-api-access-lxd5h\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:45.231962 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.231877 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:45.442672 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.442593 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8"] Sep 08 07:10:45.445738 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:10:45.445710 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a2524d1_876e_4db1_ba27_84151cdfc850.slice/crio-ee1cba7f59528016a61b5b3990e5a3c31b319c7dfebacd329c14715cd13c2842 WatchSource:0}: Error finding container ee1cba7f59528016a61b5b3990e5a3c31b319c7dfebacd329c14715cd13c2842: Status 404 returned error can't find the container with id ee1cba7f59528016a61b5b3990e5a3c31b319c7dfebacd329c14715cd13c2842 Sep 08 07:10:45.510598 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:45.510573 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6"] Sep 08 07:10:45.512422 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:10:45.512394 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f6ad42_81d6_4d8d_a0bd_21d7c5cfa596.slice/crio-07a8d79197908731cb083cc056565c38ee9c48775dec456c433ab86938e0ee55 WatchSource:0}: Error finding container 07a8d79197908731cb083cc056565c38ee9c48775dec456c433ab86938e0ee55: Status 404 returned error can't find the container with id 07a8d79197908731cb083cc056565c38ee9c48775dec456c433ab86938e0ee55 Sep 08 07:10:46.436932 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:46.436891 2571 generic.go:356] "Generic (PLEG): container finished" podID="f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" containerID="1045c33662afff5c25a4e9f078abcc47d0a4b75ea9ee5c4652a113b6b2021e4d" exitCode=0 Sep 08 07:10:46.437429 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:46.436985 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" event={"ID":"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596","Type":"ContainerDied","Data":"1045c33662afff5c25a4e9f078abcc47d0a4b75ea9ee5c4652a113b6b2021e4d"} Sep 08 07:10:46.437429 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:46.437038 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" event={"ID":"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596","Type":"ContainerStarted","Data":"07a8d79197908731cb083cc056565c38ee9c48775dec456c433ab86938e0ee55"} Sep 08 07:10:46.438619 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:46.438559 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" event={"ID":"8a2524d1-876e-4db1-ba27-84151cdfc850","Type":"ContainerStarted","Data":"ee1cba7f59528016a61b5b3990e5a3c31b319c7dfebacd329c14715cd13c2842"} Sep 08 07:10:49.457376 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:49.457337 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" event={"ID":"8a2524d1-876e-4db1-ba27-84151cdfc850","Type":"ContainerStarted","Data":"a4c675576e9f9c8c2bbe896c6253c9b600f56ef48fd9d3c8421ba1b14f0a272e"} Sep 08 07:10:49.459142 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:49.459111 2571 generic.go:356] "Generic (PLEG): container finished" podID="f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" containerID="feda291344618f6128fddacbeb47bd418cc62efb323a6af48ac2c603bfe752b8" exitCode=0 Sep 08 07:10:49.459264 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:49.459153 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" event={"ID":"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596","Type":"ContainerDied","Data":"feda291344618f6128fddacbeb47bd418cc62efb323a6af48ac2c603bfe752b8"} Sep 08 07:10:50.466154 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:50.466115 2571 generic.go:356] "Generic (PLEG): container finished" podID="f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" containerID="f4776ba7a608ace286bedff2ca957726fef26caff8132366a456ec21cacfa330" exitCode=0 Sep 08 07:10:50.466621 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:50.466191 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" event={"ID":"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596","Type":"ContainerDied","Data":"f4776ba7a608ace286bedff2ca957726fef26caff8132366a456ec21cacfa330"} Sep 08 07:10:50.467879 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:50.467856 2571 generic.go:356] "Generic (PLEG): container finished" podID="8a2524d1-876e-4db1-ba27-84151cdfc850" containerID="a4c675576e9f9c8c2bbe896c6253c9b600f56ef48fd9d3c8421ba1b14f0a272e" exitCode=0 Sep 08 07:10:50.467979 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:50.467881 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" event={"ID":"8a2524d1-876e-4db1-ba27-84151cdfc850","Type":"ContainerDied","Data":"a4c675576e9f9c8c2bbe896c6253c9b600f56ef48fd9d3c8421ba1b14f0a272e"} Sep 08 07:10:51.474440 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.474406 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" event={"ID":"8a2524d1-876e-4db1-ba27-84151cdfc850","Type":"ContainerStarted","Data":"4249b2517805d82f3551cfb3f73d3cd64ef9efc73ad83c57d88ea6e74848b019"} Sep 08 07:10:51.474851 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.474689 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:10:51.499208 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.499139 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" podStartSLOduration=2.604752 podStartE2EDuration="7.499116385s" totalImagesPullingTime="3.805897814s" totalInitContainerRuntime="1.088466571s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:10:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:10:51.495509663 +0000 UTC m=+366.173472798" watchObservedRunningTime="2026-09-08 07:10:51.499116385 +0000 UTC m=+366.177079503" Sep 08 07:10:51.603727 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.603701 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:51.744490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.744400 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-bundle\") pod \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " Sep 08 07:10:51.744634 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.744504 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-util\") pod \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " Sep 08 07:10:51.744634 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.744540 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxd5h\" (UniqueName: \"kubernetes.io/projected/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-kube-api-access-lxd5h\") pod \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\" (UID: \"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596\") " Sep 08 07:10:51.745244 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.745194 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-bundle" (OuterVolumeSpecName: "bundle") pod "f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" (UID: "f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:10:51.746703 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.746671 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-kube-api-access-lxd5h" (OuterVolumeSpecName: "kube-api-access-lxd5h") pod "f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" (UID: "f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596"). InnerVolumeSpecName "kube-api-access-lxd5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:10:51.748699 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.748666 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-util" (OuterVolumeSpecName: "util") pod "f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" (UID: "f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:10:51.846101 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.846061 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-util\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:10:51.846101 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.846093 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxd5h\" (UniqueName: \"kubernetes.io/projected/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-kube-api-access-lxd5h\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:10:51.846101 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:51.846104 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:10:52.480669 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:52.480637 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" Sep 08 07:10:52.480669 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:52.480660 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07pbl6" event={"ID":"f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596","Type":"ContainerDied","Data":"07a8d79197908731cb083cc056565c38ee9c48775dec456c433ab86938e0ee55"} Sep 08 07:10:52.481101 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:10:52.480709 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07a8d79197908731cb083cc056565c38ee9c48775dec456c433ab86938e0ee55" Sep 08 07:11:02.483563 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:02.483530 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79c9ffb96f-2bss8" Sep 08 07:11:06.495677 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.495636 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bd9f9987c-9c2q8" podUID="3ccb3186-601f-467c-b51b-99639536e65a" containerName="console" containerID="cri-o://150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244" gracePeriod=15 Sep 08 07:11:06.747356 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.747293 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:11:06.757649 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.757624 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-console-config\") pod \"3ccb3186-601f-467c-b51b-99639536e65a\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " Sep 08 07:11:06.757739 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.757677 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-service-ca\") pod \"3ccb3186-601f-467c-b51b-99639536e65a\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " Sep 08 07:11:06.757739 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.757703 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tt2q7\" (UniqueName: \"kubernetes.io/projected/3ccb3186-601f-467c-b51b-99639536e65a-kube-api-access-tt2q7\") pod \"3ccb3186-601f-467c-b51b-99639536e65a\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " Sep 08 07:11:06.757739 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.757726 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-trusted-ca-bundle\") pod \"3ccb3186-601f-467c-b51b-99639536e65a\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " Sep 08 07:11:06.757872 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.757751 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-serving-cert\") pod \"3ccb3186-601f-467c-b51b-99639536e65a\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " Sep 08 07:11:06.757872 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.757809 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-oauth-config\") pod \"3ccb3186-601f-467c-b51b-99639536e65a\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " Sep 08 07:11:06.757872 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.757843 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-oauth-serving-cert\") pod \"3ccb3186-601f-467c-b51b-99639536e65a\" (UID: \"3ccb3186-601f-467c-b51b-99639536e65a\") " Sep 08 07:11:06.758141 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.758105 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-console-config" (OuterVolumeSpecName: "console-config") pod "3ccb3186-601f-467c-b51b-99639536e65a" (UID: "3ccb3186-601f-467c-b51b-99639536e65a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:11:06.758141 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.758107 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-service-ca" (OuterVolumeSpecName: "service-ca") pod "3ccb3186-601f-467c-b51b-99639536e65a" (UID: "3ccb3186-601f-467c-b51b-99639536e65a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:11:06.758327 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.758162 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3ccb3186-601f-467c-b51b-99639536e65a" (UID: "3ccb3186-601f-467c-b51b-99639536e65a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:11:06.758519 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.758493 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3ccb3186-601f-467c-b51b-99639536e65a" (UID: "3ccb3186-601f-467c-b51b-99639536e65a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:11:06.760011 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.759985 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ccb3186-601f-467c-b51b-99639536e65a-kube-api-access-tt2q7" (OuterVolumeSpecName: "kube-api-access-tt2q7") pod "3ccb3186-601f-467c-b51b-99639536e65a" (UID: "3ccb3186-601f-467c-b51b-99639536e65a"). InnerVolumeSpecName "kube-api-access-tt2q7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:11:06.760104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.760032 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3ccb3186-601f-467c-b51b-99639536e65a" (UID: "3ccb3186-601f-467c-b51b-99639536e65a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:11:06.760428 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.760403 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3ccb3186-601f-467c-b51b-99639536e65a" (UID: "3ccb3186-601f-467c-b51b-99639536e65a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:11:06.858510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.858469 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-oauth-config\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:11:06.858510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.858502 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-oauth-serving-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:11:06.858510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.858512 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-console-config\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:11:06.858510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.858520 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:11:06.858774 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.858528 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt2q7\" (UniqueName: \"kubernetes.io/projected/3ccb3186-601f-467c-b51b-99639536e65a-kube-api-access-tt2q7\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:11:06.858774 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.858537 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ccb3186-601f-467c-b51b-99639536e65a-trusted-ca-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:11:06.858774 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:06.858545 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ccb3186-601f-467c-b51b-99639536e65a-console-serving-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:11:07.548800 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:07.548756 2571 generic.go:356] "Generic (PLEG): container finished" podID="3ccb3186-601f-467c-b51b-99639536e65a" containerID="150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244" exitCode=0 Sep 08 07:11:07.548800 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:07.548803 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bd9f9987c-9c2q8" Sep 08 07:11:07.549296 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:07.548843 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bd9f9987c-9c2q8" event={"ID":"3ccb3186-601f-467c-b51b-99639536e65a","Type":"ContainerDied","Data":"150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244"} Sep 08 07:11:07.549296 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:07.548902 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bd9f9987c-9c2q8" event={"ID":"3ccb3186-601f-467c-b51b-99639536e65a","Type":"ContainerDied","Data":"8f5c600814399d8a3438712091f5309fc49ee3d0827f87b520bcb8e88017d60c"} Sep 08 07:11:07.549296 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:07.548922 2571 scope.go:117] "RemoveContainer" containerID="150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244" Sep 08 07:11:07.557706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:07.557684 2571 scope.go:117] "RemoveContainer" containerID="150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244" Sep 08 07:11:07.557971 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:11:07.557953 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244\": container with ID starting with 150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244 not found: ID does not exist" containerID="150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244" Sep 08 07:11:07.558023 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:07.557979 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244"} err="failed to get container status \"150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244\": rpc error: code = NotFound desc = could not find container \"150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244\": container with ID starting with 150aca301aa69e1fc22796bc0ab429871298c90920c678e5e4d135a2fe6f6244 not found: ID does not exist" Sep 08 07:11:07.570745 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:07.570708 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bd9f9987c-9c2q8"] Sep 08 07:11:07.572226 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:07.572189 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bd9f9987c-9c2q8"] Sep 08 07:11:07.807897 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:07.807817 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ccb3186-601f-467c-b51b-99639536e65a" path="/var/lib/kubelet/pods/3ccb3186-601f-467c-b51b-99639536e65a/volumes" Sep 08 07:11:15.068768 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.068731 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6cwdg"] Sep 08 07:11:15.069335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.069046 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ccb3186-601f-467c-b51b-99639536e65a" containerName="console" Sep 08 07:11:15.069335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.069059 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ccb3186-601f-467c-b51b-99639536e65a" containerName="console" Sep 08 07:11:15.069335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.069070 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" containerName="extract" Sep 08 07:11:15.069335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.069076 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" containerName="extract" Sep 08 07:11:15.069335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.069093 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" containerName="pull" Sep 08 07:11:15.069335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.069099 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" containerName="pull" Sep 08 07:11:15.069335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.069116 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" containerName="util" Sep 08 07:11:15.069335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.069121 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" containerName="util" Sep 08 07:11:15.069335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.069179 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ccb3186-601f-467c-b51b-99639536e65a" containerName="console" Sep 08 07:11:15.069335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.069189 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3f6ad42-81d6-4d8d-a0bd-21d7c5cfa596" containerName="extract" Sep 08 07:11:15.072098 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.072081 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" Sep 08 07:11:15.074511 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.074480 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 08 07:11:15.074511 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.074484 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mcqkr\"" Sep 08 07:11:15.081716 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.081692 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6cwdg"] Sep 08 07:11:15.123717 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.123688 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ccc54f6-419a-4e85-be30-00100141ee31-cert\") pod \"odh-model-controller-7998c67dc4-6cwdg\" (UID: \"5ccc54f6-419a-4e85-be30-00100141ee31\") " pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" Sep 08 07:11:15.123876 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.123740 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-267hx\" (UniqueName: \"kubernetes.io/projected/5ccc54f6-419a-4e85-be30-00100141ee31-kube-api-access-267hx\") pod \"odh-model-controller-7998c67dc4-6cwdg\" (UID: \"5ccc54f6-419a-4e85-be30-00100141ee31\") " pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" Sep 08 07:11:15.224261 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.224199 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-267hx\" (UniqueName: \"kubernetes.io/projected/5ccc54f6-419a-4e85-be30-00100141ee31-kube-api-access-267hx\") pod \"odh-model-controller-7998c67dc4-6cwdg\" (UID: \"5ccc54f6-419a-4e85-be30-00100141ee31\") " pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" Sep 08 07:11:15.224489 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.224342 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ccc54f6-419a-4e85-be30-00100141ee31-cert\") pod \"odh-model-controller-7998c67dc4-6cwdg\" (UID: \"5ccc54f6-419a-4e85-be30-00100141ee31\") " pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" Sep 08 07:11:15.226801 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.226764 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ccc54f6-419a-4e85-be30-00100141ee31-cert\") pod \"odh-model-controller-7998c67dc4-6cwdg\" (UID: \"5ccc54f6-419a-4e85-be30-00100141ee31\") " pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" Sep 08 07:11:15.232811 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.232784 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-267hx\" (UniqueName: \"kubernetes.io/projected/5ccc54f6-419a-4e85-be30-00100141ee31-kube-api-access-267hx\") pod \"odh-model-controller-7998c67dc4-6cwdg\" (UID: \"5ccc54f6-419a-4e85-be30-00100141ee31\") " pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" Sep 08 07:11:15.382813 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.382735 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" Sep 08 07:11:15.650298 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.650199 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6qbws"] Sep 08 07:11:15.653829 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.653806 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:11:15.656632 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.656605 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-w7nkn\"" Sep 08 07:11:15.657020 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.656997 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 08 07:11:15.659975 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.659954 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6cwdg"] Sep 08 07:11:15.662136 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:11:15.662110 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ccc54f6_419a_4e85_be30_00100141ee31.slice/crio-c600231b3f31dfb793b383a15af40d71aae7ec6153f38dc0b5a407b168ad49a5 WatchSource:0}: Error finding container c600231b3f31dfb793b383a15af40d71aae7ec6153f38dc0b5a407b168ad49a5: Status 404 returned error can't find the container with id c600231b3f31dfb793b383a15af40d71aae7ec6153f38dc0b5a407b168ad49a5 Sep 08 07:11:15.667183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.667161 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6qbws"] Sep 08 07:11:15.728933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.728892 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4b16240-e668-43ca-a967-2649668f1951-tls-certs\") pod \"model-serving-api-54d89bd5fc-6qbws\" (UID: \"d4b16240-e668-43ca-a967-2649668f1951\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:11:15.729094 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.728977 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcd4l\" (UniqueName: \"kubernetes.io/projected/d4b16240-e668-43ca-a967-2649668f1951-kube-api-access-dcd4l\") pod \"model-serving-api-54d89bd5fc-6qbws\" (UID: \"d4b16240-e668-43ca-a967-2649668f1951\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:11:15.829499 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.829467 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcd4l\" (UniqueName: \"kubernetes.io/projected/d4b16240-e668-43ca-a967-2649668f1951-kube-api-access-dcd4l\") pod \"model-serving-api-54d89bd5fc-6qbws\" (UID: \"d4b16240-e668-43ca-a967-2649668f1951\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:11:15.829700 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.829624 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4b16240-e668-43ca-a967-2649668f1951-tls-certs\") pod \"model-serving-api-54d89bd5fc-6qbws\" (UID: \"d4b16240-e668-43ca-a967-2649668f1951\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:11:15.829766 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:11:15.829754 2571 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Sep 08 07:11:15.829846 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:11:15.829827 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4b16240-e668-43ca-a967-2649668f1951-tls-certs podName:d4b16240-e668-43ca-a967-2649668f1951 nodeName:}" failed. No retries permitted until 2026-09-08 07:11:16.329806374 +0000 UTC m=+391.007769481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d4b16240-e668-43ca-a967-2649668f1951-tls-certs") pod "model-serving-api-54d89bd5fc-6qbws" (UID: "d4b16240-e668-43ca-a967-2649668f1951") : secret "model-serving-api-tls" not found Sep 08 07:11:15.839238 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:15.839195 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcd4l\" (UniqueName: \"kubernetes.io/projected/d4b16240-e668-43ca-a967-2649668f1951-kube-api-access-dcd4l\") pod \"model-serving-api-54d89bd5fc-6qbws\" (UID: \"d4b16240-e668-43ca-a967-2649668f1951\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:11:16.088200 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.088167 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk"] Sep 08 07:11:16.092010 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.091985 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.094575 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.094550 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 08 07:11:16.094881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.094862 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 08 07:11:16.095001 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.094862 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-67nv6\"" Sep 08 07:11:16.095001 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.094981 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 08 07:11:16.104061 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.104026 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk"] Sep 08 07:11:16.132004 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.131967 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.132004 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.132005 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.132248 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.132029 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhqdl\" (UniqueName: \"kubernetes.io/projected/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-kube-api-access-mhqdl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.132248 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.132096 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.132248 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.132125 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.132248 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.132144 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.132470 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.132277 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.132470 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.132355 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.132470 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.132391 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233111 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233077 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233133 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233157 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233193 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhqdl\" (UniqueName: \"kubernetes.io/projected/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-kube-api-access-mhqdl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233250 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233275 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233303 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233611 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233366 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233611 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233415 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233611 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233577 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233763 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233672 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.233921 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233879 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.234038 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233921 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.234038 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.233995 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.236457 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.236430 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.236684 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.236661 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.242507 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.242430 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.242507 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.242448 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhqdl\" (UniqueName: \"kubernetes.io/projected/d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc-kube-api-access-mhqdl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk\" (UID: \"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.334775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.334735 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4b16240-e668-43ca-a967-2649668f1951-tls-certs\") pod \"model-serving-api-54d89bd5fc-6qbws\" (UID: \"d4b16240-e668-43ca-a967-2649668f1951\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:11:16.337427 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.337393 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4b16240-e668-43ca-a967-2649668f1951-tls-certs\") pod \"model-serving-api-54d89bd5fc-6qbws\" (UID: \"d4b16240-e668-43ca-a967-2649668f1951\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:11:16.404009 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.403920 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:16.574378 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.574334 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:11:16.606114 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.606076 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" event={"ID":"5ccc54f6-419a-4e85-be30-00100141ee31","Type":"ContainerStarted","Data":"c600231b3f31dfb793b383a15af40d71aae7ec6153f38dc0b5a407b168ad49a5"} Sep 08 07:11:16.693455 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.693421 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk"] Sep 08 07:11:16.693993 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:11:16.693965 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e92058_a9fc_4f2c_a52e_13abb9d5ebcc.slice/crio-86cc15e45f5e3e19a399ac025beba260edc0177f3da48b72984b068830aeb177 WatchSource:0}: Error finding container 86cc15e45f5e3e19a399ac025beba260edc0177f3da48b72984b068830aeb177: Status 404 returned error can't find the container with id 86cc15e45f5e3e19a399ac025beba260edc0177f3da48b72984b068830aeb177 Sep 08 07:11:16.859282 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:16.859246 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6qbws"] Sep 08 07:11:16.860549 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:11:16.860517 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b16240_e668_43ca_a967_2649668f1951.slice/crio-b5ecb3cacb8573c81f88b899b1ceedf6a69ed76011a61555a0dd127414e0a9d1 WatchSource:0}: Error finding container b5ecb3cacb8573c81f88b899b1ceedf6a69ed76011a61555a0dd127414e0a9d1: Status 404 returned error can't find the container with id b5ecb3cacb8573c81f88b899b1ceedf6a69ed76011a61555a0dd127414e0a9d1 Sep 08 07:11:17.613665 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:17.613623 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" event={"ID":"d4b16240-e668-43ca-a967-2649668f1951","Type":"ContainerStarted","Data":"b5ecb3cacb8573c81f88b899b1ceedf6a69ed76011a61555a0dd127414e0a9d1"} Sep 08 07:11:17.615067 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:17.615038 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" event={"ID":"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc","Type":"ContainerStarted","Data":"86cc15e45f5e3e19a399ac025beba260edc0177f3da48b72984b068830aeb177"} Sep 08 07:11:20.904422 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:20.904373 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:11:20.904695 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:20.904457 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:11:20.904695 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:20.904491 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:11:21.639198 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:21.639154 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" event={"ID":"d4b16240-e668-43ca-a967-2649668f1951","Type":"ContainerStarted","Data":"d027a8780e1671ff1bedddac62864423bc9cb99a30780417a7b07166e6a2ccba"} Sep 08 07:11:21.639417 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:21.639297 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:11:21.640872 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:21.640834 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" event={"ID":"d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc","Type":"ContainerStarted","Data":"6c6572e3cbed6806cfd2e1dd4c102a0f6979ddcd34d50d92b02ba772fe11c94b"} Sep 08 07:11:21.642880 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:21.642821 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" event={"ID":"5ccc54f6-419a-4e85-be30-00100141ee31","Type":"ContainerStarted","Data":"3c17fe802f078c414ee6896d24786fa57e301b0ada8882123897da4823946af1"} Sep 08 07:11:21.642996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:21.642948 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" Sep 08 07:11:21.676946 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:21.676887 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" podStartSLOduration=2.628162167 podStartE2EDuration="6.676866739s" totalImagesPullingTime="4.048704572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:11:21.658558398 +0000 UTC m=+396.336521515" watchObservedRunningTime="2026-09-08 07:11:21.676866739 +0000 UTC m=+396.354829885" Sep 08 07:11:21.677422 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:21.677380 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" podStartSLOduration=1.445290951 podStartE2EDuration="6.677367893s" totalImagesPullingTime="5.232076942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:11:21.675680658 +0000 UTC m=+396.353643797" watchObservedRunningTime="2026-09-08 07:11:21.677367893 +0000 UTC m=+396.355331006" Sep 08 07:11:21.700900 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:21.700849 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" podStartSLOduration=1.493523133 podStartE2EDuration="5.700831177s" totalImagesPullingTime="4.207308044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:11:21.69810486 +0000 UTC m=+396.376067997" watchObservedRunningTime="2026-09-08 07:11:21.700831177 +0000 UTC m=+396.378794292" Sep 08 07:11:22.008209 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.008170 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq"] Sep 08 07:11:22.078201 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.078160 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq"] Sep 08 07:11:22.078382 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.078298 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.081343 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.081130 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 08 07:11:22.081343 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.081169 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 08 07:11:22.081343 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.081204 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 08 07:11:22.081343 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.081242 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7gfcm\"" Sep 08 07:11:22.188281 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.188248 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.188281 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.188283 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cdd1284b-171c-44ac-8344-fbdfe35898f9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.188558 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.188303 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.188558 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.188383 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.188558 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.188418 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmlnw\" (UniqueName: \"kubernetes.io/projected/cdd1284b-171c-44ac-8344-fbdfe35898f9-kube-api-access-gmlnw\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.188558 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.188462 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdd1284b-171c-44ac-8344-fbdfe35898f9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.188558 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.188514 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.289422 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.289334 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.289422 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.289393 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.289422 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.289418 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cdd1284b-171c-44ac-8344-fbdfe35898f9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.289697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.289444 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.289697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.289507 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.289697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.289534 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmlnw\" (UniqueName: \"kubernetes.io/projected/cdd1284b-171c-44ac-8344-fbdfe35898f9-kube-api-access-gmlnw\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.289697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.289601 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdd1284b-171c-44ac-8344-fbdfe35898f9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.290200 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.290173 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.291865 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.291836 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdd1284b-171c-44ac-8344-fbdfe35898f9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.291865 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.291857 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.292249 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.292191 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cdd1284b-171c-44ac-8344-fbdfe35898f9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.292325 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.292262 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.301567 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.301545 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdd1284b-171c-44ac-8344-fbdfe35898f9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.301840 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.301819 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmlnw\" (UniqueName: \"kubernetes.io/projected/cdd1284b-171c-44ac-8344-fbdfe35898f9-kube-api-access-gmlnw\") pod \"istiod-openshift-gateway-5677ccb49b-wh4dq\" (UID: \"cdd1284b-171c-44ac-8344-fbdfe35898f9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.390535 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.390493 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:22.404654 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.404620 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:22.409198 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.409172 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:22.647807 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.647726 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:22.648822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.648797 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk" Sep 08 07:11:22.698638 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:22.698614 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq"] Sep 08 07:11:22.700000 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:11:22.699974 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdd1284b_171c_44ac_8344_fbdfe35898f9.slice/crio-e71ddd6779d0cbe44e250f20d0d1e5956dfad59ffdd13bf28f699d7fd49f6841 WatchSource:0}: Error finding container e71ddd6779d0cbe44e250f20d0d1e5956dfad59ffdd13bf28f699d7fd49f6841: Status 404 returned error can't find the container with id e71ddd6779d0cbe44e250f20d0d1e5956dfad59ffdd13bf28f699d7fd49f6841 Sep 08 07:11:23.660256 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:23.660184 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" event={"ID":"cdd1284b-171c-44ac-8344-fbdfe35898f9","Type":"ContainerStarted","Data":"e71ddd6779d0cbe44e250f20d0d1e5956dfad59ffdd13bf28f699d7fd49f6841"} Sep 08 07:11:25.492673 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:25.492621 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:11:25.493090 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:25.492714 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:11:25.672171 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:25.672138 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" event={"ID":"cdd1284b-171c-44ac-8344-fbdfe35898f9","Type":"ContainerStarted","Data":"fe424f2bffa1dc00f6fc7cf5be524a9da49764492b393bab18a7b47702115788"} Sep 08 07:11:25.672384 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:25.672271 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:26.678971 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:26.678935 2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-wh4dq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 07:11:26.679355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:26.678987 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" podUID="cdd1284b-171c-44ac-8344-fbdfe35898f9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 07:11:29.678139 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:29.678105 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" Sep 08 07:11:29.711412 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:29.711357 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wh4dq" podStartSLOduration=5.921173578 podStartE2EDuration="8.711341191s" totalImagesPullingTime="2.790167613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:11:25.704206148 +0000 UTC m=+400.382169263" watchObservedRunningTime="2026-09-08 07:11:29.711341191 +0000 UTC m=+404.389304307" Sep 08 07:11:32.650714 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:32.650678 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-6cwdg" Sep 08 07:11:32.652489 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:11:32.652469 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6qbws" Sep 08 07:12:19.291466 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.291389 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6"] Sep 08 07:12:19.295075 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.295056 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.297406 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.297377 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6b6kz\"" Sep 08 07:12:19.297520 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.297493 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 07:12:19.298460 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.298443 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 07:12:19.304086 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.304065 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6"] Sep 08 07:12:19.367369 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.367336 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.367530 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.367397 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtg7p\" (UniqueName: \"kubernetes.io/projected/8ca7dc5e-c01e-4eff-a38f-93235df45063-kube-api-access-dtg7p\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.367530 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.367423 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.391928 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.391892 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr"] Sep 08 07:12:19.395399 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.395380 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.404103 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.404078 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr"] Sep 08 07:12:19.467902 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.467869 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.468073 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.467918 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.468073 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.467970 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtg7p\" (UniqueName: \"kubernetes.io/projected/8ca7dc5e-c01e-4eff-a38f-93235df45063-kube-api-access-dtg7p\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.468073 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.467990 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.468073 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.468015 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.468202 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.468076 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh4v2\" (UniqueName: \"kubernetes.io/projected/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-kube-api-access-nh4v2\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.468328 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.468300 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.468420 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.468342 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.478002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.477976 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtg7p\" (UniqueName: \"kubernetes.io/projected/8ca7dc5e-c01e-4eff-a38f-93235df45063-kube-api-access-dtg7p\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.493140 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.493107 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk"] Sep 08 07:12:19.496654 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.496635 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.504348 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.504324 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk"] Sep 08 07:12:19.569016 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.568922 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh4v2\" (UniqueName: \"kubernetes.io/projected/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-kube-api-access-nh4v2\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.569016 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.568977 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzf8w\" (UniqueName: \"kubernetes.io/projected/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-kube-api-access-wzf8w\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.569016 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.569017 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.569330 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.569059 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.569330 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.569107 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.569330 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.569136 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.569453 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.569430 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.569489 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.569454 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.577696 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.577669 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh4v2\" (UniqueName: \"kubernetes.io/projected/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-kube-api-access-nh4v2\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.591480 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.591451 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk"] Sep 08 07:12:19.595092 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.595076 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.603485 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.603462 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk"] Sep 08 07:12:19.604836 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.604817 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:19.670015 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.669845 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzf8w\" (UniqueName: \"kubernetes.io/projected/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-kube-api-access-wzf8w\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.670015 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.669985 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.670312 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.670018 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.670312 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.670051 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.670312 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.670090 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.670312 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.670142 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plg95\" (UniqueName: \"kubernetes.io/projected/daf995a6-381c-446d-95fe-3bb8a1078aac-kube-api-access-plg95\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.670652 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.670628 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.670707 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.670680 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.680961 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.680933 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzf8w\" (UniqueName: \"kubernetes.io/projected/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-kube-api-access-wzf8w\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.705570 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.705544 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:19.770829 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.770795 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plg95\" (UniqueName: \"kubernetes.io/projected/daf995a6-381c-446d-95fe-3bb8a1078aac-kube-api-access-plg95\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.770987 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.770887 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.770987 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.770907 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.771337 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.771316 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.771337 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.771331 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.779356 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.779330 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plg95\" (UniqueName: \"kubernetes.io/projected/daf995a6-381c-446d-95fe-3bb8a1078aac-kube-api-access-plg95\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.808821 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.807301 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:19.877510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.877487 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6"] Sep 08 07:12:19.879359 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:12:19.879333 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca7dc5e_c01e_4eff_a38f_93235df45063.slice/crio-3e7abc8241ca8f0b1c610b12dddbdcd2a9b341a4c09549efa18e9fe9e32c283a WatchSource:0}: Error finding container 3e7abc8241ca8f0b1c610b12dddbdcd2a9b341a4c09549efa18e9fe9e32c283a: Status 404 returned error can't find the container with id 3e7abc8241ca8f0b1c610b12dddbdcd2a9b341a4c09549efa18e9fe9e32c283a Sep 08 07:12:19.905303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.905276 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:19.927133 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.927099 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" event={"ID":"8ca7dc5e-c01e-4eff-a38f-93235df45063","Type":"ContainerStarted","Data":"3e7abc8241ca8f0b1c610b12dddbdcd2a9b341a4c09549efa18e9fe9e32c283a"} Sep 08 07:12:19.978134 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:19.978107 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr"] Sep 08 07:12:19.979963 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:12:19.979933 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d051e2_24ee_45b2_9b1d_cd3178a5a7b2.slice/crio-3901ce9b931b7cda9ea8671bbad501a7696d46487f5dd4d87c7bc84051744682 WatchSource:0}: Error finding container 3901ce9b931b7cda9ea8671bbad501a7696d46487f5dd4d87c7bc84051744682: Status 404 returned error can't find the container with id 3901ce9b931b7cda9ea8671bbad501a7696d46487f5dd4d87c7bc84051744682 Sep 08 07:12:20.078451 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.078421 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk"] Sep 08 07:12:20.082114 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:12:20.082076 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117d87fc_7aeb_4e59_ba6f_487c3cd9bb02.slice/crio-ca582826013137578560e538f66a4fe604e4adab6ff879d224594c75ae02e3c6 WatchSource:0}: Error finding container ca582826013137578560e538f66a4fe604e4adab6ff879d224594c75ae02e3c6: Status 404 returned error can't find the container with id ca582826013137578560e538f66a4fe604e4adab6ff879d224594c75ae02e3c6 Sep 08 07:12:20.177243 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.177202 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk"] Sep 08 07:12:20.205297 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:12:20.205267 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf995a6_381c_446d_95fe_3bb8a1078aac.slice/crio-0678b517824a0a8b151ea1ea617ba2ad6e8d5b6633a70cb2bb2fe6954ebfd780 WatchSource:0}: Error finding container 0678b517824a0a8b151ea1ea617ba2ad6e8d5b6633a70cb2bb2fe6954ebfd780: Status 404 returned error can't find the container with id 0678b517824a0a8b151ea1ea617ba2ad6e8d5b6633a70cb2bb2fe6954ebfd780 Sep 08 07:12:20.933869 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.933832 2571 generic.go:356] "Generic (PLEG): container finished" podID="8ca7dc5e-c01e-4eff-a38f-93235df45063" containerID="f6f52f9df8752e37fbbe0feda1ee21c4a3cc94870b400626abecc344a234b2ca" exitCode=0 Sep 08 07:12:20.934330 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.933911 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" event={"ID":"8ca7dc5e-c01e-4eff-a38f-93235df45063","Type":"ContainerDied","Data":"f6f52f9df8752e37fbbe0feda1ee21c4a3cc94870b400626abecc344a234b2ca"} Sep 08 07:12:20.935580 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.935556 2571 generic.go:356] "Generic (PLEG): container finished" podID="08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" containerID="2ec1f0598b594fb3949f7080683b33ae4fd2f1559b7448e1aa30f6fba305f2aa" exitCode=0 Sep 08 07:12:20.935952 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.935638 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" event={"ID":"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2","Type":"ContainerDied","Data":"2ec1f0598b594fb3949f7080683b33ae4fd2f1559b7448e1aa30f6fba305f2aa"} Sep 08 07:12:20.935952 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.935686 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" event={"ID":"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2","Type":"ContainerStarted","Data":"3901ce9b931b7cda9ea8671bbad501a7696d46487f5dd4d87c7bc84051744682"} Sep 08 07:12:20.937358 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.937334 2571 generic.go:356] "Generic (PLEG): container finished" podID="117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" containerID="ee5cfe37dde7a4c4f09a0ad15c056029a35d79754c269ff1cbd3b9fd494af8d2" exitCode=0 Sep 08 07:12:20.937508 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.937408 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" event={"ID":"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02","Type":"ContainerDied","Data":"ee5cfe37dde7a4c4f09a0ad15c056029a35d79754c269ff1cbd3b9fd494af8d2"} Sep 08 07:12:20.937508 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.937444 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" event={"ID":"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02","Type":"ContainerStarted","Data":"ca582826013137578560e538f66a4fe604e4adab6ff879d224594c75ae02e3c6"} Sep 08 07:12:20.942256 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.942234 2571 generic.go:356] "Generic (PLEG): container finished" podID="daf995a6-381c-446d-95fe-3bb8a1078aac" containerID="8652f6b64f20c59df45cee105ee4a1a627cf18e7ff4cb54762e6447a6ed51c9b" exitCode=0 Sep 08 07:12:20.942355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.942260 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" event={"ID":"daf995a6-381c-446d-95fe-3bb8a1078aac","Type":"ContainerDied","Data":"8652f6b64f20c59df45cee105ee4a1a627cf18e7ff4cb54762e6447a6ed51c9b"} Sep 08 07:12:20.942355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:20.942300 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" event={"ID":"daf995a6-381c-446d-95fe-3bb8a1078aac","Type":"ContainerStarted","Data":"0678b517824a0a8b151ea1ea617ba2ad6e8d5b6633a70cb2bb2fe6954ebfd780"} Sep 08 07:12:22.953150 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:22.953116 2571 generic.go:356] "Generic (PLEG): container finished" podID="daf995a6-381c-446d-95fe-3bb8a1078aac" containerID="0bfb6c4b5d83abd1cd83fb905109ad865fb94214fc17e17fa28c1881817dbd68" exitCode=0 Sep 08 07:12:22.953619 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:22.953194 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" event={"ID":"daf995a6-381c-446d-95fe-3bb8a1078aac","Type":"ContainerDied","Data":"0bfb6c4b5d83abd1cd83fb905109ad865fb94214fc17e17fa28c1881817dbd68"} Sep 08 07:12:22.955195 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:22.955171 2571 generic.go:356] "Generic (PLEG): container finished" podID="8ca7dc5e-c01e-4eff-a38f-93235df45063" containerID="f89f43404f0d3eec2671772c992638ab7f7a72449cac3ccf6d00ae141a77f626" exitCode=0 Sep 08 07:12:22.955268 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:22.955247 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" event={"ID":"8ca7dc5e-c01e-4eff-a38f-93235df45063","Type":"ContainerDied","Data":"f89f43404f0d3eec2671772c992638ab7f7a72449cac3ccf6d00ae141a77f626"} Sep 08 07:12:22.957105 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:22.957086 2571 generic.go:356] "Generic (PLEG): container finished" podID="08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" containerID="30f53c2c24da9494c0021980faf496b4ce5557483fa861f13657d6db75f9d31e" exitCode=0 Sep 08 07:12:22.957203 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:22.957165 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" event={"ID":"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2","Type":"ContainerDied","Data":"30f53c2c24da9494c0021980faf496b4ce5557483fa861f13657d6db75f9d31e"} Sep 08 07:12:22.959074 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:22.959054 2571 generic.go:356] "Generic (PLEG): container finished" podID="117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" containerID="83c11e7358a3aa972282dd6d5056838bd466be137acff8e11cc2dec4f488ad1c" exitCode=0 Sep 08 07:12:22.959150 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:22.959093 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" event={"ID":"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02","Type":"ContainerDied","Data":"83c11e7358a3aa972282dd6d5056838bd466be137acff8e11cc2dec4f488ad1c"} Sep 08 07:12:23.965705 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:23.965670 2571 generic.go:356] "Generic (PLEG): container finished" podID="daf995a6-381c-446d-95fe-3bb8a1078aac" containerID="ed1114f86384cfd53edc8ac38d3b0d1ee9e69ddea22d632f0111e8844ed1f411" exitCode=0 Sep 08 07:12:23.966138 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:23.965747 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" event={"ID":"daf995a6-381c-446d-95fe-3bb8a1078aac","Type":"ContainerDied","Data":"ed1114f86384cfd53edc8ac38d3b0d1ee9e69ddea22d632f0111e8844ed1f411"} Sep 08 07:12:23.970914 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:23.970888 2571 generic.go:356] "Generic (PLEG): container finished" podID="8ca7dc5e-c01e-4eff-a38f-93235df45063" containerID="96d962032b883aaf86873b87049798e60a35fc5b8888234aaecb6c51bbe12eb3" exitCode=0 Sep 08 07:12:23.971064 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:23.970977 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" event={"ID":"8ca7dc5e-c01e-4eff-a38f-93235df45063","Type":"ContainerDied","Data":"96d962032b883aaf86873b87049798e60a35fc5b8888234aaecb6c51bbe12eb3"} Sep 08 07:12:23.973092 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:23.973073 2571 generic.go:356] "Generic (PLEG): container finished" podID="08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" containerID="1d7b6e2956e6fce3778f072b2fb729c122f62fdf53185169498c3ce6f2076312" exitCode=0 Sep 08 07:12:23.973198 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:23.973148 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" event={"ID":"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2","Type":"ContainerDied","Data":"1d7b6e2956e6fce3778f072b2fb729c122f62fdf53185169498c3ce6f2076312"} Sep 08 07:12:23.974964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:23.974947 2571 generic.go:356] "Generic (PLEG): container finished" podID="117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" containerID="855b1727d0f395c5058ac8a9a03c927fbb0007507229a549e0f2731d53174cd1" exitCode=0 Sep 08 07:12:23.975044 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:23.974998 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" event={"ID":"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02","Type":"ContainerDied","Data":"855b1727d0f395c5058ac8a9a03c927fbb0007507229a549e0f2731d53174cd1"} Sep 08 07:12:25.117118 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.117085 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:12:25.157621 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.157592 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:25.161862 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.161841 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:25.180007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.179979 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:25.221759 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.221723 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-bundle\") pod \"daf995a6-381c-446d-95fe-3bb8a1078aac\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " Sep 08 07:12:25.221933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.221780 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-util\") pod \"daf995a6-381c-446d-95fe-3bb8a1078aac\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " Sep 08 07:12:25.221933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.221845 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzf8w\" (UniqueName: \"kubernetes.io/projected/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-kube-api-access-wzf8w\") pod \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " Sep 08 07:12:25.221933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.221872 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-bundle\") pod \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " Sep 08 07:12:25.221933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.221921 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plg95\" (UniqueName: \"kubernetes.io/projected/daf995a6-381c-446d-95fe-3bb8a1078aac-kube-api-access-plg95\") pod \"daf995a6-381c-446d-95fe-3bb8a1078aac\" (UID: \"daf995a6-381c-446d-95fe-3bb8a1078aac\") " Sep 08 07:12:25.222158 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.221952 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-util\") pod \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\" (UID: \"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02\") " Sep 08 07:12:25.222158 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.221992 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtg7p\" (UniqueName: \"kubernetes.io/projected/8ca7dc5e-c01e-4eff-a38f-93235df45063-kube-api-access-dtg7p\") pod \"8ca7dc5e-c01e-4eff-a38f-93235df45063\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " Sep 08 07:12:25.222158 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.222033 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-bundle\") pod \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " Sep 08 07:12:25.222158 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.222069 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-bundle\") pod \"8ca7dc5e-c01e-4eff-a38f-93235df45063\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " Sep 08 07:12:25.222158 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.222116 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-util\") pod \"8ca7dc5e-c01e-4eff-a38f-93235df45063\" (UID: \"8ca7dc5e-c01e-4eff-a38f-93235df45063\") " Sep 08 07:12:25.222158 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.222142 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh4v2\" (UniqueName: \"kubernetes.io/projected/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-kube-api-access-nh4v2\") pod \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " Sep 08 07:12:25.222511 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.222206 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-util\") pod \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\" (UID: \"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2\") " Sep 08 07:12:25.222609 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.222581 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-bundle" (OuterVolumeSpecName: "bundle") pod "117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" (UID: "117d87fc-7aeb-4e59-ba6f-487c3cd9bb02"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:12:25.222837 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.222780 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-bundle" (OuterVolumeSpecName: "bundle") pod "daf995a6-381c-446d-95fe-3bb8a1078aac" (UID: "daf995a6-381c-446d-95fe-3bb8a1078aac"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:12:25.223441 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.223107 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-bundle" (OuterVolumeSpecName: "bundle") pod "8ca7dc5e-c01e-4eff-a38f-93235df45063" (UID: "8ca7dc5e-c01e-4eff-a38f-93235df45063"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:12:25.223441 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.223425 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-bundle" (OuterVolumeSpecName: "bundle") pod "08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" (UID: "08d051e2-24ee-45b2-9b1d-cd3178a5a7b2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:12:25.225044 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.225023 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-kube-api-access-wzf8w" (OuterVolumeSpecName: "kube-api-access-wzf8w") pod "117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" (UID: "117d87fc-7aeb-4e59-ba6f-487c3cd9bb02"). InnerVolumeSpecName "kube-api-access-wzf8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:12:25.225143 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.225062 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daf995a6-381c-446d-95fe-3bb8a1078aac-kube-api-access-plg95" (OuterVolumeSpecName: "kube-api-access-plg95") pod "daf995a6-381c-446d-95fe-3bb8a1078aac" (UID: "daf995a6-381c-446d-95fe-3bb8a1078aac"). InnerVolumeSpecName "kube-api-access-plg95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:12:25.225380 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.225352 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ca7dc5e-c01e-4eff-a38f-93235df45063-kube-api-access-dtg7p" (OuterVolumeSpecName: "kube-api-access-dtg7p") pod "8ca7dc5e-c01e-4eff-a38f-93235df45063" (UID: "8ca7dc5e-c01e-4eff-a38f-93235df45063"). InnerVolumeSpecName "kube-api-access-dtg7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:12:25.225666 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.225648 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-kube-api-access-nh4v2" (OuterVolumeSpecName: "kube-api-access-nh4v2") pod "08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" (UID: "08d051e2-24ee-45b2-9b1d-cd3178a5a7b2"). InnerVolumeSpecName "kube-api-access-nh4v2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:12:25.228351 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.228318 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-util" (OuterVolumeSpecName: "util") pod "daf995a6-381c-446d-95fe-3bb8a1078aac" (UID: "daf995a6-381c-446d-95fe-3bb8a1078aac"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:12:25.322946 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.322867 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-util\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.322946 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.322898 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzf8w\" (UniqueName: \"kubernetes.io/projected/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-kube-api-access-wzf8w\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.322946 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.322910 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.322946 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.322920 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plg95\" (UniqueName: \"kubernetes.io/projected/daf995a6-381c-446d-95fe-3bb8a1078aac-kube-api-access-plg95\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.322946 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.322929 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtg7p\" (UniqueName: \"kubernetes.io/projected/8ca7dc5e-c01e-4eff-a38f-93235df45063-kube-api-access-dtg7p\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.322946 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.322938 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.322946 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.322947 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.323300 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.322955 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh4v2\" (UniqueName: \"kubernetes.io/projected/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-kube-api-access-nh4v2\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.323300 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.322963 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/daf995a6-381c-446d-95fe-3bb8a1078aac-bundle\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.530451 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.530385 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-util" (OuterVolumeSpecName: "util") pod "117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" (UID: "117d87fc-7aeb-4e59-ba6f-487c3cd9bb02"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:12:25.536788 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.536761 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-util" (OuterVolumeSpecName: "util") pod "08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" (UID: "08d051e2-24ee-45b2-9b1d-cd3178a5a7b2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:12:25.537070 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.537050 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-util" (OuterVolumeSpecName: "util") pod "8ca7dc5e-c01e-4eff-a38f-93235df45063" (UID: "8ca7dc5e-c01e-4eff-a38f-93235df45063"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:12:25.625360 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.625274 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ca7dc5e-c01e-4eff-a38f-93235df45063-util\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.625360 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.625303 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08d051e2-24ee-45b2-9b1d-cd3178a5a7b2-util\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.625360 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.625312 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/117d87fc-7aeb-4e59-ba6f-487c3cd9bb02-util\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:12:25.991789 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.991761 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" Sep 08 07:12:25.991789 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.991764 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0bdzw6" event={"ID":"8ca7dc5e-c01e-4eff-a38f-93235df45063","Type":"ContainerDied","Data":"3e7abc8241ca8f0b1c610b12dddbdcd2a9b341a4c09549efa18e9fe9e32c283a"} Sep 08 07:12:25.992001 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.991833 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e7abc8241ca8f0b1c610b12dddbdcd2a9b341a4c09549efa18e9fe9e32c283a" Sep 08 07:12:25.993854 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.993809 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" Sep 08 07:12:25.993854 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.993836 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c94ghr" event={"ID":"08d051e2-24ee-45b2-9b1d-cd3178a5a7b2","Type":"ContainerDied","Data":"3901ce9b931b7cda9ea8671bbad501a7696d46487f5dd4d87c7bc84051744682"} Sep 08 07:12:25.994022 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.993897 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3901ce9b931b7cda9ea8671bbad501a7696d46487f5dd4d87c7bc84051744682" Sep 08 07:12:25.995932 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.995913 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" Sep 08 07:12:25.996036 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.995949 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb662vk" event={"ID":"117d87fc-7aeb-4e59-ba6f-487c3cd9bb02","Type":"ContainerDied","Data":"ca582826013137578560e538f66a4fe604e4adab6ff879d224594c75ae02e3c6"} Sep 08 07:12:25.996036 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.995989 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca582826013137578560e538f66a4fe604e4adab6ff879d224594c75ae02e3c6" Sep 08 07:12:25.998035 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.998011 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" event={"ID":"daf995a6-381c-446d-95fe-3bb8a1078aac","Type":"ContainerDied","Data":"0678b517824a0a8b151ea1ea617ba2ad6e8d5b6633a70cb2bb2fe6954ebfd780"} Sep 08 07:12:25.998168 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.998051 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0678b517824a0a8b151ea1ea617ba2ad6e8d5b6633a70cb2bb2fe6954ebfd780" Sep 08 07:12:25.998168 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:12:25.998053 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053dz8nk" Sep 08 07:13:38.344313 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344226 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm"] Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344694 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" containerName="util" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344709 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" containerName="util" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344719 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daf995a6-381c-446d-95fe-3bb8a1078aac" containerName="extract" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344726 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="daf995a6-381c-446d-95fe-3bb8a1078aac" containerName="extract" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344739 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca7dc5e-c01e-4eff-a38f-93235df45063" containerName="util" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344747 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca7dc5e-c01e-4eff-a38f-93235df45063" containerName="util" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344757 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca7dc5e-c01e-4eff-a38f-93235df45063" containerName="pull" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344762 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca7dc5e-c01e-4eff-a38f-93235df45063" containerName="pull" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344768 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca7dc5e-c01e-4eff-a38f-93235df45063" containerName="extract" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344773 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca7dc5e-c01e-4eff-a38f-93235df45063" containerName="extract" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344781 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" containerName="extract" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344785 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" containerName="extract" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344806 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daf995a6-381c-446d-95fe-3bb8a1078aac" containerName="util" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344814 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="daf995a6-381c-446d-95fe-3bb8a1078aac" containerName="util" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344825 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" containerName="util" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344833 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" containerName="util" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344842 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daf995a6-381c-446d-95fe-3bb8a1078aac" containerName="pull" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344847 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="daf995a6-381c-446d-95fe-3bb8a1078aac" containerName="pull" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344853 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" containerName="pull" Sep 08 07:13:38.344843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344858 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" containerName="pull" Sep 08 07:13:38.345831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344870 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" containerName="pull" Sep 08 07:13:38.345831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344875 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" containerName="pull" Sep 08 07:13:38.345831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344896 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" containerName="extract" Sep 08 07:13:38.345831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344905 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" containerName="extract" Sep 08 07:13:38.345831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344960 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="117d87fc-7aeb-4e59-ba6f-487c3cd9bb02" containerName="extract" Sep 08 07:13:38.345831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344969 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="daf995a6-381c-446d-95fe-3bb8a1078aac" containerName="extract" Sep 08 07:13:38.345831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344977 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d051e2-24ee-45b2-9b1d-cd3178a5a7b2" containerName="extract" Sep 08 07:13:38.345831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.344989 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ca7dc5e-c01e-4eff-a38f-93235df45063" containerName="extract" Sep 08 07:13:38.347976 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.347957 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.350753 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.350720 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-28894\"" Sep 08 07:13:38.360173 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.360147 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm"] Sep 08 07:13:38.458682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.458647 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.458852 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.458696 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bb0f622-eca8-4bfd-9663-a720190f39c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.458852 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.458730 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.458852 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.458769 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.458852 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.458823 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.458996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.458858 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.458996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.458886 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.458996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.458903 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.458996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.458973 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lzx7\" (UniqueName: \"kubernetes.io/projected/6bb0f622-eca8-4bfd-9663-a720190f39c9-kube-api-access-2lzx7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560027 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.559994 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560208 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560036 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560208 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560054 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560208 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560085 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560426 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560206 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560426 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560300 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lzx7\" (UniqueName: \"kubernetes.io/projected/6bb0f622-eca8-4bfd-9663-a720190f39c9-kube-api-access-2lzx7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560426 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560371 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560426 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560402 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bb0f622-eca8-4bfd-9663-a720190f39c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560619 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560444 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560619 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560494 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560619 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560453 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560619 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560544 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.560837 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.560664 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.561346 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.561302 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bb0f622-eca8-4bfd-9663-a720190f39c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.563116 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.563072 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.563303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.563282 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.568432 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.568412 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bb0f622-eca8-4bfd-9663-a720190f39c9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.568768 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.568743 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lzx7\" (UniqueName: \"kubernetes.io/projected/6bb0f622-eca8-4bfd-9663-a720190f39c9-kube-api-access-2lzx7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdwbm\" (UID: \"6bb0f622-eca8-4bfd-9663-a720190f39c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.659674 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.659588 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:38.937870 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.937841 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm"] Sep 08 07:13:38.939773 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:13:38.939747 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb0f622_eca8_4bfd_9663_a720190f39c9.slice/crio-a94c9463edd9c6f0eff454e66c83869cd81c09c03739f5cbd3eeacc217315eca WatchSource:0}: Error finding container a94c9463edd9c6f0eff454e66c83869cd81c09c03739f5cbd3eeacc217315eca: Status 404 returned error can't find the container with id a94c9463edd9c6f0eff454e66c83869cd81c09c03739f5cbd3eeacc217315eca Sep 08 07:13:38.942338 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.942302 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:13:38.942423 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.942372 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:13:38.942423 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:38.942402 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:13:39.360072 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:39.360034 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" event={"ID":"6bb0f622-eca8-4bfd-9663-a720190f39c9","Type":"ContainerStarted","Data":"3f6ac9d676993151a3dbce660649598d006362d9feaca9097575c02a2b1ce4ca"} Sep 08 07:13:39.360488 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:39.360117 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" event={"ID":"6bb0f622-eca8-4bfd-9663-a720190f39c9","Type":"ContainerStarted","Data":"a94c9463edd9c6f0eff454e66c83869cd81c09c03739f5cbd3eeacc217315eca"} Sep 08 07:13:39.382619 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:39.382567 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" podStartSLOduration=1.382550964 podStartE2EDuration="1.382550964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:13:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:13:39.380672032 +0000 UTC m=+534.058635176" watchObservedRunningTime="2026-09-08 07:13:39.382550964 +0000 UTC m=+534.060514079" Sep 08 07:13:39.660357 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:39.660260 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:39.665148 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:39.665120 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:40.365271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:40.365240 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:13:40.366338 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:13:40.366318 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:14:25.887505 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:25.887470 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f96498768-nc98h"] Sep 08 07:14:25.891006 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:25.890979 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:25.893469 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:25.893444 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 08 07:14:25.893593 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:25.893491 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 08 07:14:25.893720 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:25.893698 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 08 07:14:25.893786 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:25.893706 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w2vnv\"" Sep 08 07:14:25.901856 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:25.901827 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f96498768-nc98h"] Sep 08 07:14:25.978159 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:25.978124 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-webhook-cert\") pod \"maas-controller-f96498768-nc98h\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:25.978380 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:25.978184 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-metrics-certs\") pod \"maas-controller-f96498768-nc98h\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:25.978380 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:25.978248 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wqbb\" (UniqueName: \"kubernetes.io/projected/901b073c-ade8-4f32-8593-b973d9bf704a-kube-api-access-4wqbb\") pod \"maas-controller-f96498768-nc98h\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:26.078949 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:26.078910 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wqbb\" (UniqueName: \"kubernetes.io/projected/901b073c-ade8-4f32-8593-b973d9bf704a-kube-api-access-4wqbb\") pod \"maas-controller-f96498768-nc98h\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:26.079123 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:26.078992 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-webhook-cert\") pod \"maas-controller-f96498768-nc98h\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:26.079123 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:26.079027 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-metrics-certs\") pod \"maas-controller-f96498768-nc98h\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:26.079209 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:14:26.079127 2571 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 08 07:14:26.079209 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:14:26.079195 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-webhook-cert podName:901b073c-ade8-4f32-8593-b973d9bf704a nodeName:}" failed. No retries permitted until 2026-09-08 07:14:26.579178564 +0000 UTC m=+581.257141658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-webhook-cert") pod "maas-controller-f96498768-nc98h" (UID: "901b073c-ade8-4f32-8593-b973d9bf704a") : secret "maas-controller-webhook-cert" not found Sep 08 07:14:26.081431 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:26.081413 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-metrics-certs\") pod \"maas-controller-f96498768-nc98h\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:26.092839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:26.092811 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wqbb\" (UniqueName: \"kubernetes.io/projected/901b073c-ade8-4f32-8593-b973d9bf704a-kube-api-access-4wqbb\") pod \"maas-controller-f96498768-nc98h\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:26.584453 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:26.584418 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-webhook-cert\") pod \"maas-controller-f96498768-nc98h\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:26.586745 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:26.586715 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-webhook-cert\") pod \"maas-controller-f96498768-nc98h\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:26.803053 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:26.803017 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:27.074533 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:27.074501 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f96498768-nc98h"] Sep 08 07:14:27.076569 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:14:27.076540 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod901b073c_ade8_4f32_8593_b973d9bf704a.slice/crio-745908a3598dffdb3205c46f8ea285d3530b54e837439cefe743dfc21eadf3c3 WatchSource:0}: Error finding container 745908a3598dffdb3205c46f8ea285d3530b54e837439cefe743dfc21eadf3c3: Status 404 returned error can't find the container with id 745908a3598dffdb3205c46f8ea285d3530b54e837439cefe743dfc21eadf3c3 Sep 08 07:14:27.608290 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:27.608243 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f96498768-nc98h" event={"ID":"901b073c-ade8-4f32-8593-b973d9bf704a","Type":"ContainerStarted","Data":"745908a3598dffdb3205c46f8ea285d3530b54e837439cefe743dfc21eadf3c3"} Sep 08 07:14:29.620787 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:29.620683 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f96498768-nc98h" event={"ID":"901b073c-ade8-4f32-8593-b973d9bf704a","Type":"ContainerStarted","Data":"f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee"} Sep 08 07:14:29.621169 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:29.620786 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:29.640000 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:29.639928 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f96498768-nc98h" podStartSLOduration=2.445050556 podStartE2EDuration="4.639908141s" totalImagesPullingTime="2.194857585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:14:29.63894462 +0000 UTC m=+584.316907760" watchObservedRunningTime="2026-09-08 07:14:29.639908141 +0000 UTC m=+584.317871280" Sep 08 07:14:40.438574 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.438536 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f96498768-nc98h"] Sep 08 07:14:40.439104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.438775 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-f96498768-nc98h" podUID="901b073c-ade8-4f32-8593-b973d9bf704a" containerName="manager" containerID="cri-o://f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee" gracePeriod=30 Sep 08 07:14:40.440744 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.440719 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:40.678317 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.678287 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:40.679858 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.679834 2571 generic.go:356] "Generic (PLEG): container finished" podID="901b073c-ade8-4f32-8593-b973d9bf704a" containerID="f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee" exitCode=0 Sep 08 07:14:40.679983 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.679873 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f96498768-nc98h" Sep 08 07:14:40.679983 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.679917 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f96498768-nc98h" event={"ID":"901b073c-ade8-4f32-8593-b973d9bf704a","Type":"ContainerDied","Data":"f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee"} Sep 08 07:14:40.679983 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.679971 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f96498768-nc98h" event={"ID":"901b073c-ade8-4f32-8593-b973d9bf704a","Type":"ContainerDied","Data":"745908a3598dffdb3205c46f8ea285d3530b54e837439cefe743dfc21eadf3c3"} Sep 08 07:14:40.680097 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.680000 2571 scope.go:117] "RemoveContainer" containerID="f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee" Sep 08 07:14:40.688276 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.688240 2571 scope.go:117] "RemoveContainer" containerID="f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee" Sep 08 07:14:40.688565 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:14:40.688547 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee\": container with ID starting with f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee not found: ID does not exist" containerID="f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee" Sep 08 07:14:40.688663 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.688579 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee"} err="failed to get container status \"f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee\": rpc error: code = NotFound desc = could not find container \"f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee\": container with ID starting with f3b601fce88463ad42cd25a8b1b35b6d14ac743c52bb4e9fe0e8b4d29d1b57ee not found: ID does not exist" Sep 08 07:14:40.708441 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.708410 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-webhook-cert\") pod \"901b073c-ade8-4f32-8593-b973d9bf704a\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " Sep 08 07:14:40.708615 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.708514 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-metrics-certs\") pod \"901b073c-ade8-4f32-8593-b973d9bf704a\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " Sep 08 07:14:40.708615 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.708575 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wqbb\" (UniqueName: \"kubernetes.io/projected/901b073c-ade8-4f32-8593-b973d9bf704a-kube-api-access-4wqbb\") pod \"901b073c-ade8-4f32-8593-b973d9bf704a\" (UID: \"901b073c-ade8-4f32-8593-b973d9bf704a\") " Sep 08 07:14:40.710770 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.710735 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "901b073c-ade8-4f32-8593-b973d9bf704a" (UID: "901b073c-ade8-4f32-8593-b973d9bf704a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:14:40.710916 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.710788 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/901b073c-ade8-4f32-8593-b973d9bf704a-kube-api-access-4wqbb" (OuterVolumeSpecName: "kube-api-access-4wqbb") pod "901b073c-ade8-4f32-8593-b973d9bf704a" (UID: "901b073c-ade8-4f32-8593-b973d9bf704a"). InnerVolumeSpecName "kube-api-access-4wqbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:14:40.710916 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.710846 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "901b073c-ade8-4f32-8593-b973d9bf704a" (UID: "901b073c-ade8-4f32-8593-b973d9bf704a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:14:40.809438 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.809390 2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-metrics-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:14:40.809438 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.809431 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wqbb\" (UniqueName: \"kubernetes.io/projected/901b073c-ade8-4f32-8593-b973d9bf704a-kube-api-access-4wqbb\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:14:40.809438 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:40.809441 2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/901b073c-ade8-4f32-8593-b973d9bf704a-webhook-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:14:41.003579 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:41.003543 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f96498768-nc98h"] Sep 08 07:14:41.008790 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:41.008764 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f96498768-nc98h"] Sep 08 07:14:41.808722 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:14:41.808686 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="901b073c-ade8-4f32-8593-b973d9bf704a" path="/var/lib/kubelet/pods/901b073c-ade8-4f32-8593-b973d9bf704a/volumes" Sep 08 07:15:02.956851 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.956807 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b"] Sep 08 07:15:02.961822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.957307 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="901b073c-ade8-4f32-8593-b973d9bf704a" containerName="manager" Sep 08 07:15:02.961822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.957330 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="901b073c-ade8-4f32-8593-b973d9bf704a" containerName="manager" Sep 08 07:15:02.961822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.957409 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="901b073c-ade8-4f32-8593-b973d9bf704a" containerName="manager" Sep 08 07:15:02.962385 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.962366 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" Sep 08 07:15:02.965279 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.965177 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jh94h\"" Sep 08 07:15:02.965279 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.965206 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 08 07:15:02.966669 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.966646 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-wwl9h"] Sep 08 07:15:02.973771 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.973748 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b"] Sep 08 07:15:02.973890 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.973871 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" Sep 08 07:15:02.981808 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:02.981781 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-wwl9h"] Sep 08 07:15:03.113185 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.113146 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c23d559-3fc3-4c49-bd81-bcc28552a895-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-wcf8b\" (UID: \"8c23d559-3fc3-4c49-bd81-bcc28552a895\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" Sep 08 07:15:03.113395 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.113241 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxkl9\" (UniqueName: \"kubernetes.io/projected/8c23d559-3fc3-4c49-bd81-bcc28552a895-kube-api-access-rxkl9\") pod \"payload-pre-processing-59fbc47bf8-wcf8b\" (UID: \"8c23d559-3fc3-4c49-bd81-bcc28552a895\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" Sep 08 07:15:03.113395 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.113268 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tzkj\" (UniqueName: \"kubernetes.io/projected/9f88a82d-71c8-4d94-a9ff-e93e5b7f704c-kube-api-access-4tzkj\") pod \"payload-processing-5b54b7fbb7-wwl9h\" (UID: \"9f88a82d-71c8-4d94-a9ff-e93e5b7f704c\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" Sep 08 07:15:03.113395 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.113299 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f88a82d-71c8-4d94-a9ff-e93e5b7f704c-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-wwl9h\" (UID: \"9f88a82d-71c8-4d94-a9ff-e93e5b7f704c\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" Sep 08 07:15:03.213958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.213858 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxkl9\" (UniqueName: \"kubernetes.io/projected/8c23d559-3fc3-4c49-bd81-bcc28552a895-kube-api-access-rxkl9\") pod \"payload-pre-processing-59fbc47bf8-wcf8b\" (UID: \"8c23d559-3fc3-4c49-bd81-bcc28552a895\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" Sep 08 07:15:03.213958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.213906 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tzkj\" (UniqueName: \"kubernetes.io/projected/9f88a82d-71c8-4d94-a9ff-e93e5b7f704c-kube-api-access-4tzkj\") pod \"payload-processing-5b54b7fbb7-wwl9h\" (UID: \"9f88a82d-71c8-4d94-a9ff-e93e5b7f704c\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" Sep 08 07:15:03.213958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.213954 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f88a82d-71c8-4d94-a9ff-e93e5b7f704c-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-wwl9h\" (UID: \"9f88a82d-71c8-4d94-a9ff-e93e5b7f704c\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" Sep 08 07:15:03.214274 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.214085 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c23d559-3fc3-4c49-bd81-bcc28552a895-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-wcf8b\" (UID: \"8c23d559-3fc3-4c49-bd81-bcc28552a895\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" Sep 08 07:15:03.215009 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.214970 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f88a82d-71c8-4d94-a9ff-e93e5b7f704c-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-wwl9h\" (UID: \"9f88a82d-71c8-4d94-a9ff-e93e5b7f704c\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" Sep 08 07:15:03.215138 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.215040 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c23d559-3fc3-4c49-bd81-bcc28552a895-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-wcf8b\" (UID: \"8c23d559-3fc3-4c49-bd81-bcc28552a895\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" Sep 08 07:15:03.225060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.225021 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxkl9\" (UniqueName: \"kubernetes.io/projected/8c23d559-3fc3-4c49-bd81-bcc28552a895-kube-api-access-rxkl9\") pod \"payload-pre-processing-59fbc47bf8-wcf8b\" (UID: \"8c23d559-3fc3-4c49-bd81-bcc28552a895\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" Sep 08 07:15:03.225488 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.225463 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tzkj\" (UniqueName: \"kubernetes.io/projected/9f88a82d-71c8-4d94-a9ff-e93e5b7f704c-kube-api-access-4tzkj\") pod \"payload-processing-5b54b7fbb7-wwl9h\" (UID: \"9f88a82d-71c8-4d94-a9ff-e93e5b7f704c\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" Sep 08 07:15:03.274315 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.274272 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" Sep 08 07:15:03.285838 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.285806 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" Sep 08 07:15:03.568347 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.568315 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b"] Sep 08 07:15:03.571594 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:15:03.571529 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c23d559_3fc3_4c49_bd81_bcc28552a895.slice/crio-323bde34e2c744b8517d72b5fe1df02de89ba3db523ba186749b37b18817c446 WatchSource:0}: Error finding container 323bde34e2c744b8517d72b5fe1df02de89ba3db523ba186749b37b18817c446: Status 404 returned error can't find the container with id 323bde34e2c744b8517d72b5fe1df02de89ba3db523ba186749b37b18817c446 Sep 08 07:15:03.572189 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.572162 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-wwl9h"] Sep 08 07:15:03.575527 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.573616 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 07:15:03.575527 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:15:03.575481 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f88a82d_71c8_4d94_a9ff_e93e5b7f704c.slice/crio-0b3dd62a0f83d5448abdd73b597e36e98c410194d5b77ba592fd05ebee8c0ef8 WatchSource:0}: Error finding container 0b3dd62a0f83d5448abdd73b597e36e98c410194d5b77ba592fd05ebee8c0ef8: Status 404 returned error can't find the container with id 0b3dd62a0f83d5448abdd73b597e36e98c410194d5b77ba592fd05ebee8c0ef8 Sep 08 07:15:03.800392 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.800343 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" event={"ID":"9f88a82d-71c8-4d94-a9ff-e93e5b7f704c","Type":"ContainerStarted","Data":"0b3dd62a0f83d5448abdd73b597e36e98c410194d5b77ba592fd05ebee8c0ef8"} Sep 08 07:15:03.801649 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:03.801624 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" event={"ID":"8c23d559-3fc3-4c49-bd81-bcc28552a895","Type":"ContainerStarted","Data":"323bde34e2c744b8517d72b5fe1df02de89ba3db523ba186749b37b18817c446"} Sep 08 07:15:06.823827 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:06.823791 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" event={"ID":"8c23d559-3fc3-4c49-bd81-bcc28552a895","Type":"ContainerStarted","Data":"6abe99d75269a8fbbbf84fb50ced6f5a32825d7cf5e75c4f3e7b41f1e11584e7"} Sep 08 07:15:06.824302 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:06.824030 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" Sep 08 07:15:06.825548 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:06.825519 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" event={"ID":"9f88a82d-71c8-4d94-a9ff-e93e5b7f704c","Type":"ContainerStarted","Data":"3c2e274aa747103638b76b70f52c2e83937da40344f9b77cd7d04e4289c65935"} Sep 08 07:15:06.825655 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:06.825621 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" Sep 08 07:15:06.840264 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:06.840169 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" podStartSLOduration=2.592917741 podStartE2EDuration="4.840149978s" totalImagesPullingTime="2.247232237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:15:06.839841131 +0000 UTC m=+621.517804265" watchObservedRunningTime="2026-09-08 07:15:06.840149978 +0000 UTC m=+621.518113097" Sep 08 07:15:06.856765 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:06.856702 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" podStartSLOduration=2.607211611 podStartE2EDuration="4.856684804s" totalImagesPullingTime="2.249473193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:15:06.854172244 +0000 UTC m=+621.532135380" watchObservedRunningTime="2026-09-08 07:15:06.856684804 +0000 UTC m=+621.534647950" Sep 08 07:15:11.407401 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:11.407353 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdwbm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 08 07:15:11.407799 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:11.407429 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" podUID="6bb0f622-eca8-4bfd-9663-a720190f39c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 07:15:11.853779 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:11.853746 2571 generic.go:356] "Generic (PLEG): container finished" podID="6bb0f622-eca8-4bfd-9663-a720190f39c9" containerID="3f6ac9d676993151a3dbce660649598d006362d9feaca9097575c02a2b1ce4ca" exitCode=0 Sep 08 07:15:11.854020 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:11.853815 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" event={"ID":"6bb0f622-eca8-4bfd-9663-a720190f39c9","Type":"ContainerDied","Data":"3f6ac9d676993151a3dbce660649598d006362d9feaca9097575c02a2b1ce4ca"} Sep 08 07:15:11.854199 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:11.854185 2571 scope.go:117] "RemoveContainer" containerID="3f6ac9d676993151a3dbce660649598d006362d9feaca9097575c02a2b1ce4ca" Sep 08 07:15:11.855201 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:11.855172 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:15:11.855336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:11.855257 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:15:11.855336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:11.855292 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:15:12.861142 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:12.861103 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" event={"ID":"6bb0f622-eca8-4bfd-9663-a720190f39c9","Type":"ContainerStarted","Data":"25b7e58e99947613c08f9617f832436c50ea0f7edad8629c59881e32f65c20a3"} Sep 08 07:15:13.660595 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:13.660559 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:15:13.660778 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:13.660760 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Sep 08 07:15:13.660838 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:13.660818 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" podUID="6bb0f622-eca8-4bfd-9663-a720190f39c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Sep 08 07:15:14.660824 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:14.660786 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Sep 08 07:15:14.661250 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:14.660847 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" podUID="6bb0f622-eca8-4bfd-9663-a720190f39c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Sep 08 07:15:15.659964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:15.659924 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Sep 08 07:15:15.660140 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:15.659984 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" podUID="6bb0f622-eca8-4bfd-9663-a720190f39c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Sep 08 07:15:16.660754 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:16.660714 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Sep 08 07:15:16.661144 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:16.660775 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" podUID="6bb0f622-eca8-4bfd-9663-a720190f39c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Sep 08 07:15:17.660380 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:17.660346 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdwbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Sep 08 07:15:17.660637 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:17.660402 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" podUID="6bb0f622-eca8-4bfd-9663-a720190f39c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Sep 08 07:15:17.833277 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:17.833239 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-wwl9h" Sep 08 07:15:17.833649 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:17.833323 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wcf8b" Sep 08 07:15:18.664034 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:18.663990 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:15:18.664456 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:18.664427 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:15:18.665032 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:18.665014 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdwbm" Sep 08 07:15:55.643772 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.643739 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns"] Sep 08 07:15:55.647427 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.647405 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.651084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.651059 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 08 07:15:55.651084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.651080 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 08 07:15:55.651304 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.651059 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zg7w8\"" Sep 08 07:15:55.651304 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.651072 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 08 07:15:55.658651 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.658623 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns"] Sep 08 07:15:55.776790 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.776757 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.776964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.776829 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.776964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.776872 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.776964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.776920 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.776964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.776944 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npnbw\" (UniqueName: \"kubernetes.io/projected/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-kube-api-access-npnbw\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.776964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.776961 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.877500 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.877469 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npnbw\" (UniqueName: \"kubernetes.io/projected/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-kube-api-access-npnbw\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.877684 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.877506 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.877684 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.877551 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.877684 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.877599 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.877684 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.877639 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.877684 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.877680 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.877974 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.877951 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.878044 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.877993 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.878122 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.878100 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.879975 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.879948 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.880093 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.880078 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.885260 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.885238 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npnbw\" (UniqueName: \"kubernetes.io/projected/b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5-kube-api-access-npnbw\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns\" (UID: \"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:55.959130 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:55.959097 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:15:56.236125 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:56.236097 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns"] Sep 08 07:15:56.237977 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:15:56.237950 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94817e7_26f8_4f07_8bc8_8a0bdd3c1ce5.slice/crio-3c2725832e4975107df85a16c1ad113df8e6a334ebbf2967ea8baf77163ca2b0 WatchSource:0}: Error finding container 3c2725832e4975107df85a16c1ad113df8e6a334ebbf2967ea8baf77163ca2b0: Status 404 returned error can't find the container with id 3c2725832e4975107df85a16c1ad113df8e6a334ebbf2967ea8baf77163ca2b0 Sep 08 07:15:57.104302 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:57.104260 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" event={"ID":"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5","Type":"ContainerStarted","Data":"3c2725832e4975107df85a16c1ad113df8e6a334ebbf2967ea8baf77163ca2b0"} Sep 08 07:15:58.110716 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:58.110681 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" event={"ID":"b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5","Type":"ContainerStarted","Data":"c62f74115b206620de58bf2b5adb689cea13f91ae12e926509fd26c01636bbc2"} Sep 08 07:15:58.130749 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:15:58.130687 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" podStartSLOduration=2.141752106 podStartE2EDuration="3.130663719s" totalImagesPullingTime="988.911613ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:15:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:15:58.129752945 +0000 UTC m=+672.807716083" watchObservedRunningTime="2026-09-08 07:15:58.130663719 +0000 UTC m=+672.808626837" Sep 08 07:16:05.959995 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:05.959951 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:16:05.960402 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:05.960016 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:16:05.973093 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:05.973066 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:16:06.164340 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:06.164311 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns" Sep 08 07:16:19.136443 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.136408 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh"] Sep 08 07:16:19.140055 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.140035 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.142808 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.142787 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 08 07:16:19.149628 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.149602 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh"] Sep 08 07:16:19.181315 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.181282 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.181445 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.181326 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.181445 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.181381 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.181445 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.181420 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bdae08d-581c-4435-8d39-d5b53ae69435-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.181445 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.181442 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.181612 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.181460 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2vpj\" (UniqueName: \"kubernetes.io/projected/2bdae08d-581c-4435-8d39-d5b53ae69435-kube-api-access-z2vpj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.282632 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.282592 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bdae08d-581c-4435-8d39-d5b53ae69435-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.282783 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.282640 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.282783 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.282661 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2vpj\" (UniqueName: \"kubernetes.io/projected/2bdae08d-581c-4435-8d39-d5b53ae69435-kube-api-access-z2vpj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.282783 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.282684 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.282783 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.282721 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.282783 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.282757 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.283134 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.283111 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.283179 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.283160 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.283267 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.283170 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.284983 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.284958 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bdae08d-581c-4435-8d39-d5b53ae69435-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.285134 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.285115 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bdae08d-581c-4435-8d39-d5b53ae69435-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.290843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.290818 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2vpj\" (UniqueName: \"kubernetes.io/projected/2bdae08d-581c-4435-8d39-d5b53ae69435-kube-api-access-z2vpj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh\" (UID: \"2bdae08d-581c-4435-8d39-d5b53ae69435\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.451425 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.451389 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:19.725169 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:19.725137 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh"] Sep 08 07:16:19.726378 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:16:19.726355 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bdae08d_581c_4435_8d39_d5b53ae69435.slice/crio-e51e5ec7e5987ce752233d28b52e8a941864473291f17ea245e21fdc47d38f15 WatchSource:0}: Error finding container e51e5ec7e5987ce752233d28b52e8a941864473291f17ea245e21fdc47d38f15: Status 404 returned error can't find the container with id e51e5ec7e5987ce752233d28b52e8a941864473291f17ea245e21fdc47d38f15 Sep 08 07:16:20.225311 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:20.225271 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" event={"ID":"2bdae08d-581c-4435-8d39-d5b53ae69435","Type":"ContainerStarted","Data":"434265e2281c0f9c20b7b1d87440f274d284cd7f110db48afa3d28f37a4fb2fc"} Sep 08 07:16:20.225689 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:20.225335 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" event={"ID":"2bdae08d-581c-4435-8d39-d5b53ae69435","Type":"ContainerStarted","Data":"e51e5ec7e5987ce752233d28b52e8a941864473291f17ea245e21fdc47d38f15"} Sep 08 07:16:20.244048 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:20.243995 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" podStartSLOduration=1.057565 podStartE2EDuration="1.24397865s" totalImagesPullingTime="186.41365ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:16:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:16:20.24174709 +0000 UTC m=+694.919710218" watchObservedRunningTime="2026-09-08 07:16:20.24397865 +0000 UTC m=+694.921941816" Sep 08 07:16:29.452087 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:29.452046 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:29.452716 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:29.452134 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:29.465998 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:29.465968 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:16:30.287824 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:16:30.287791 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh" Sep 08 07:18:19.521824 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.521735 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7"] Sep 08 07:18:19.524957 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.524934 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.527323 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.527300 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-620112-openshift-default-dockercfg-2v4s9\"" Sep 08 07:18:19.537967 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.537946 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7"] Sep 08 07:18:19.677609 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.677569 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-data\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.677791 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.677623 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-certs\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.677791 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.677647 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w966b\" (UniqueName: \"kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-kube-api-access-w966b\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.677791 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.677699 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istiod-ca-cert\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.677898 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.677800 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-socket\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.677898 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.677848 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-credential-socket\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.677898 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.677879 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-token\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.678014 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.677905 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-envoy\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.678014 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.677929 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-podinfo\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779227 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779123 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-socket\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779227 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779163 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-credential-socket\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779227 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779182 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-token\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779267 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-envoy\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779301 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-podinfo\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779360 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-data\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779403 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-certs\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779431 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w966b\" (UniqueName: \"kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-kube-api-access-w966b\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779510 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779484 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istiod-ca-cert\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779812 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779617 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-credential-socket\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779812 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779796 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-socket\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.779917 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779900 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-data\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.780017 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.779961 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-certs\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.780174 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.780158 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istiod-ca-cert\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.782079 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.782050 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-envoy\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.782348 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.782329 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-podinfo\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.787827 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.787798 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w966b\" (UniqueName: \"kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-kube-api-access-w966b\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.787911 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.787838 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-token\") pod \"e2e-inf-a-620112-openshift-default-664bd57b66-927l7\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:19.835907 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:19.835868 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:20.117935 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:20.117905 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7"] Sep 08 07:18:20.119417 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:18:20.119392 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f4e2bc_59f2_4b2f_8f8b_5a772224ab78.slice/crio-40175539ef988743fd6835e09d36679bc8ec16410bf5e8e682b810403719cb97 WatchSource:0}: Error finding container 40175539ef988743fd6835e09d36679bc8ec16410bf5e8e682b810403719cb97: Status 404 returned error can't find the container with id 40175539ef988743fd6835e09d36679bc8ec16410bf5e8e682b810403719cb97 Sep 08 07:18:20.121418 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:20.121382 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:20.121484 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:20.121453 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:20.121521 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:20.121482 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:20.854534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:20.854489 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" event={"ID":"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78","Type":"ContainerStarted","Data":"4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10"} Sep 08 07:18:20.854992 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:20.854548 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" event={"ID":"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78","Type":"ContainerStarted","Data":"40175539ef988743fd6835e09d36679bc8ec16410bf5e8e682b810403719cb97"} Sep 08 07:18:20.877827 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:20.877778 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" podStartSLOduration=1.877762563 podStartE2EDuration="1.877762563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:18:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:18:20.874817981 +0000 UTC m=+815.552781133" watchObservedRunningTime="2026-09-08 07:18:20.877762563 +0000 UTC m=+815.555725678" Sep 08 07:18:21.836770 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:21.836733 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:21.841733 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:21.841701 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:21.859744 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:21.859719 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:21.860621 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:21.860601 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:24.542805 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.542768 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z"] Sep 08 07:18:24.546362 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.546339 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:18:24.548867 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.548850 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-620112-dockercfg-ld85n\"" Sep 08 07:18:24.548954 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.548918 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-620112\"" Sep 08 07:18:24.556145 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.556119 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z"] Sep 08 07:18:24.572961 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.572926 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd"] Sep 08 07:18:24.576223 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.576193 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:18:24.594993 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.594959 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd"] Sep 08 07:18:24.726981 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.726932 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s6t7\" (UniqueName: \"kubernetes.io/projected/981ec975-bd6c-4e89-9b03-b3e2339fe0da-kube-api-access-9s6t7\") pod \"payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z\" (UID: \"981ec975-bd6c-4e89-9b03-b3e2339fe0da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:18:24.726981 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.726982 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/943a64b8-2f78-459a-b319-5cda46aaea80-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd\" (UID: \"943a64b8-2f78-459a-b319-5cda46aaea80\") " pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:18:24.727336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.727026 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj4v6\" (UniqueName: \"kubernetes.io/projected/943a64b8-2f78-459a-b319-5cda46aaea80-kube-api-access-sj4v6\") pod \"payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd\" (UID: \"943a64b8-2f78-459a-b319-5cda46aaea80\") " pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:18:24.727336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.727057 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/981ec975-bd6c-4e89-9b03-b3e2339fe0da-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z\" (UID: \"981ec975-bd6c-4e89-9b03-b3e2339fe0da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:18:24.828515 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.828404 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s6t7\" (UniqueName: \"kubernetes.io/projected/981ec975-bd6c-4e89-9b03-b3e2339fe0da-kube-api-access-9s6t7\") pod \"payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z\" (UID: \"981ec975-bd6c-4e89-9b03-b3e2339fe0da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:18:24.828515 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.828458 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/943a64b8-2f78-459a-b319-5cda46aaea80-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd\" (UID: \"943a64b8-2f78-459a-b319-5cda46aaea80\") " pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:18:24.828515 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.828479 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj4v6\" (UniqueName: \"kubernetes.io/projected/943a64b8-2f78-459a-b319-5cda46aaea80-kube-api-access-sj4v6\") pod \"payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd\" (UID: \"943a64b8-2f78-459a-b319-5cda46aaea80\") " pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:18:24.828515 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.828505 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/981ec975-bd6c-4e89-9b03-b3e2339fe0da-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z\" (UID: \"981ec975-bd6c-4e89-9b03-b3e2339fe0da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:18:24.829179 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.829156 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/943a64b8-2f78-459a-b319-5cda46aaea80-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd\" (UID: \"943a64b8-2f78-459a-b319-5cda46aaea80\") " pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:18:24.829257 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.829168 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/981ec975-bd6c-4e89-9b03-b3e2339fe0da-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z\" (UID: \"981ec975-bd6c-4e89-9b03-b3e2339fe0da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:18:24.836699 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.836679 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s6t7\" (UniqueName: \"kubernetes.io/projected/981ec975-bd6c-4e89-9b03-b3e2339fe0da-kube-api-access-9s6t7\") pod \"payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z\" (UID: \"981ec975-bd6c-4e89-9b03-b3e2339fe0da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:18:24.836794 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.836771 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj4v6\" (UniqueName: \"kubernetes.io/projected/943a64b8-2f78-459a-b319-5cda46aaea80-kube-api-access-sj4v6\") pod \"payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd\" (UID: \"943a64b8-2f78-459a-b319-5cda46aaea80\") " pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:18:24.858589 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.858559 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:18:24.887179 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:24.887155 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:18:25.136817 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:25.136789 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z"] Sep 08 07:18:25.138705 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:18:25.138675 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod981ec975_bd6c_4e89_9b03_b3e2339fe0da.slice/crio-49f6520587ab619e4b2d8bdf198679231febf368b1b3cce68d858442a6f675dc WatchSource:0}: Error finding container 49f6520587ab619e4b2d8bdf198679231febf368b1b3cce68d858442a6f675dc: Status 404 returned error can't find the container with id 49f6520587ab619e4b2d8bdf198679231febf368b1b3cce68d858442a6f675dc Sep 08 07:18:25.171588 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:25.171565 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd"] Sep 08 07:18:25.175593 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:18:25.175567 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943a64b8_2f78_459a_b319_5cda46aaea80.slice/crio-7b02a31de450156eab3748a7c4f8373d5bc83d71e67be253bae0e1c2ed88cecb WatchSource:0}: Error finding container 7b02a31de450156eab3748a7c4f8373d5bc83d71e67be253bae0e1c2ed88cecb: Status 404 returned error can't find the container with id 7b02a31de450156eab3748a7c4f8373d5bc83d71e67be253bae0e1c2ed88cecb Sep 08 07:18:25.886733 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:25.886699 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" event={"ID":"981ec975-bd6c-4e89-9b03-b3e2339fe0da","Type":"ContainerStarted","Data":"18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9"} Sep 08 07:18:25.887186 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:25.886751 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" event={"ID":"981ec975-bd6c-4e89-9b03-b3e2339fe0da","Type":"ContainerStarted","Data":"49f6520587ab619e4b2d8bdf198679231febf368b1b3cce68d858442a6f675dc"} Sep 08 07:18:25.887186 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:25.886801 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:18:25.888474 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:25.888449 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" event={"ID":"943a64b8-2f78-459a-b319-5cda46aaea80","Type":"ContainerStarted","Data":"a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492"} Sep 08 07:18:25.888595 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:25.888493 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" event={"ID":"943a64b8-2f78-459a-b319-5cda46aaea80","Type":"ContainerStarted","Data":"7b02a31de450156eab3748a7c4f8373d5bc83d71e67be253bae0e1c2ed88cecb"} Sep 08 07:18:25.888636 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:25.888600 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:18:25.906625 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:25.906583 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" podStartSLOduration=1.90656801 podStartE2EDuration="1.90656801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:18:25.90376682 +0000 UTC m=+820.581729955" watchObservedRunningTime="2026-09-08 07:18:25.90656801 +0000 UTC m=+820.584531125" Sep 08 07:18:25.920542 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:25.920503 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" podStartSLOduration=1.920490008 podStartE2EDuration="1.920490008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:18:25.918934975 +0000 UTC m=+820.596898118" watchObservedRunningTime="2026-09-08 07:18:25.920490008 +0000 UTC m=+820.598453124" Sep 08 07:18:27.409399 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.409357 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq"] Sep 08 07:18:27.413587 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.413562 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.416018 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.415996 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4a1fb56c-openshift-default-dockercfg-596n8\"" Sep 08 07:18:27.423568 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.423544 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq"] Sep 08 07:18:27.453767 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.453736 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-envoy\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.453942 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.453771 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istiod-ca-cert\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.453942 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.453800 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-podinfo\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.453942 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.453872 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk9qr\" (UniqueName: \"kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-kube-api-access-kk9qr\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.454098 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.453968 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-data\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.454098 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.454014 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-socket\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.454098 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.454047 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-token\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.454197 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.454165 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-certs\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.454267 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.454234 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-credential-socket\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.554768 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.554723 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-certs\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.554958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.554788 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-credential-socket\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.554958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.554807 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-envoy\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.554958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.554834 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istiod-ca-cert\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.554958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.554867 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-podinfo\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.554958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.554895 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk9qr\" (UniqueName: \"kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-kube-api-access-kk9qr\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.554958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.554956 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-data\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.555329 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.554989 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-socket\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.555329 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.555015 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-token\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.555442 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.555342 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-certs\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.555547 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.555521 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-data\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.555722 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.555699 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-credential-socket\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.555991 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.555969 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istiod-ca-cert\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.555991 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.555983 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-socket\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.557873 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.557625 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-envoy\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.557873 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.557830 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-podinfo\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.570554 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.570500 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-token\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.578874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.577881 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk9qr\" (UniqueName: \"kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-kube-api-access-kk9qr\") pod \"e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:27.726861 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:27.726822 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:28.002354 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:28.002327 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq"] Sep 08 07:18:28.003421 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:18:28.003396 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc173e3_c1f0_48cf_97ff_ea5784652c30.slice/crio-d51081b739faf1a3f495a999819ab9d24b44d293d6c33bf3085c1265ff4e72d2 WatchSource:0}: Error finding container d51081b739faf1a3f495a999819ab9d24b44d293d6c33bf3085c1265ff4e72d2: Status 404 returned error can't find the container with id d51081b739faf1a3f495a999819ab9d24b44d293d6c33bf3085c1265ff4e72d2 Sep 08 07:18:28.006074 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:28.006040 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:28.006173 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:28.006120 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:28.006266 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:28.006168 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:28.908783 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:28.908750 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" event={"ID":"ddc173e3-c1f0-48cf-97ff-ea5784652c30","Type":"ContainerStarted","Data":"f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f"} Sep 08 07:18:28.909161 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:28.908830 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" event={"ID":"ddc173e3-c1f0-48cf-97ff-ea5784652c30","Type":"ContainerStarted","Data":"d51081b739faf1a3f495a999819ab9d24b44d293d6c33bf3085c1265ff4e72d2"} Sep 08 07:18:28.930977 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:28.930921 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" podStartSLOduration=1.9309049759999999 podStartE2EDuration="1.930904976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:18:28.928694444 +0000 UTC m=+823.606657580" watchObservedRunningTime="2026-09-08 07:18:28.930904976 +0000 UTC m=+823.608868091" Sep 08 07:18:29.727534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:29.727498 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:29.732677 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:29.732645 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:29.914200 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:29.914159 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:29.915286 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:29.915269 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:32.119120 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.119083 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6"] Sep 08 07:18:32.122649 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.122633 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:18:32.125320 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.125296 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-4a1fb56c-dockercfg-sh4fj\"" Sep 08 07:18:32.125444 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.125348 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-4a1fb56c\"" Sep 08 07:18:32.132751 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.132728 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6"] Sep 08 07:18:32.200201 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.200167 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4hc4\" (UniqueName: \"kubernetes.io/projected/2f4aa856-7119-4683-b3b5-ff50382f99ee-kube-api-access-r4hc4\") pod \"payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6\" (UID: \"2f4aa856-7119-4683-b3b5-ff50382f99ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:18:32.200413 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.200272 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f4aa856-7119-4683-b3b5-ff50382f99ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6\" (UID: \"2f4aa856-7119-4683-b3b5-ff50382f99ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:18:32.301350 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.301308 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f4aa856-7119-4683-b3b5-ff50382f99ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6\" (UID: \"2f4aa856-7119-4683-b3b5-ff50382f99ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:18:32.301548 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.301447 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4hc4\" (UniqueName: \"kubernetes.io/projected/2f4aa856-7119-4683-b3b5-ff50382f99ee-kube-api-access-r4hc4\") pod \"payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6\" (UID: \"2f4aa856-7119-4683-b3b5-ff50382f99ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:18:32.302083 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.302056 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f4aa856-7119-4683-b3b5-ff50382f99ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6\" (UID: \"2f4aa856-7119-4683-b3b5-ff50382f99ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:18:32.310077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.310044 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4hc4\" (UniqueName: \"kubernetes.io/projected/2f4aa856-7119-4683-b3b5-ff50382f99ee-kube-api-access-r4hc4\") pod \"payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6\" (UID: \"2f4aa856-7119-4683-b3b5-ff50382f99ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:18:32.433998 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.433902 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:18:32.733169 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.733139 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6"] Sep 08 07:18:32.734797 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:18:32.734764 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4aa856_7119_4683_b3b5_ff50382f99ee.slice/crio-fae7eb52b042964e31ccbf63c935c0fbdc07f7c0757d0e5675d00d235ba7aa39 WatchSource:0}: Error finding container fae7eb52b042964e31ccbf63c935c0fbdc07f7c0757d0e5675d00d235ba7aa39: Status 404 returned error can't find the container with id fae7eb52b042964e31ccbf63c935c0fbdc07f7c0757d0e5675d00d235ba7aa39 Sep 08 07:18:32.932622 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.932583 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" event={"ID":"2f4aa856-7119-4683-b3b5-ff50382f99ee","Type":"ContainerStarted","Data":"cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f"} Sep 08 07:18:32.932768 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.932644 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" event={"ID":"2f4aa856-7119-4683-b3b5-ff50382f99ee","Type":"ContainerStarted","Data":"fae7eb52b042964e31ccbf63c935c0fbdc07f7c0757d0e5675d00d235ba7aa39"} Sep 08 07:18:32.932768 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.932682 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:18:32.952643 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:32.952596 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" podStartSLOduration=0.952578655 podStartE2EDuration="952.578655ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:18:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:18:32.948867963 +0000 UTC m=+827.626831097" watchObservedRunningTime="2026-09-08 07:18:32.952578655 +0000 UTC m=+827.630541771" Sep 08 07:18:35.297979 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.297938 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km"] Sep 08 07:18:35.305787 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.305760 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:18:35.308645 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.308622 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-638d2d\"" Sep 08 07:18:35.308752 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.308690 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-638d2d-dockercfg-nz7ps\"" Sep 08 07:18:35.310288 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.310261 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km"] Sep 08 07:18:35.332519 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.332494 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km\" (UID: \"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:18:35.332635 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.332525 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnbwp\" (UniqueName: \"kubernetes.io/projected/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-kube-api-access-xnbwp\") pod \"payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km\" (UID: \"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:18:35.433407 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.433364 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km\" (UID: \"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:18:35.433407 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.433408 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnbwp\" (UniqueName: \"kubernetes.io/projected/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-kube-api-access-xnbwp\") pod \"payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km\" (UID: \"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:18:35.434367 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.434339 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km\" (UID: \"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:18:35.441808 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.441783 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnbwp\" (UniqueName: \"kubernetes.io/projected/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-kube-api-access-xnbwp\") pod \"payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km\" (UID: \"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:18:35.618591 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.618499 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:18:35.907449 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.907419 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km"] Sep 08 07:18:35.908411 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:18:35.908384 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0f2ee4_2b38_4ba8_b30e_d098ca8d2a30.slice/crio-89e4339720f2a84d64ee745bb0898526d9db45bbfa9218007d78e607c7984a1b WatchSource:0}: Error finding container 89e4339720f2a84d64ee745bb0898526d9db45bbfa9218007d78e607c7984a1b: Status 404 returned error can't find the container with id 89e4339720f2a84d64ee745bb0898526d9db45bbfa9218007d78e607c7984a1b Sep 08 07:18:35.950569 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:35.950542 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" event={"ID":"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30","Type":"ContainerStarted","Data":"89e4339720f2a84d64ee745bb0898526d9db45bbfa9218007d78e607c7984a1b"} Sep 08 07:18:36.897736 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:36.897696 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:18:36.898867 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:36.898844 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:18:36.958543 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:36.958506 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" event={"ID":"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30","Type":"ContainerStarted","Data":"b8f83c81141ba8993a329c9921bc2bdd2e5f3edc2c343d5328eff6d04c894dcb"} Sep 08 07:18:36.958719 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:36.958626 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:18:36.992143 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:36.992083 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" podStartSLOduration=1.9920589579999999 podStartE2EDuration="1.992058958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:18:36.978592531 +0000 UTC m=+831.656555647" watchObservedRunningTime="2026-09-08 07:18:36.992058958 +0000 UTC m=+831.670022093" Sep 08 07:18:41.998124 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:41.998080 2571 generic.go:356] "Generic (PLEG): container finished" podID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerID="4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10" exitCode=0 Sep 08 07:18:41.998651 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:41.998175 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" event={"ID":"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78","Type":"ContainerDied","Data":"4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10"} Sep 08 07:18:41.998869 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:41.998833 2571 scope.go:117] "RemoveContainer" containerID="4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10" Sep 08 07:18:42.000529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:42.000476 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:42.000632 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:42.000601 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:42.000697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:42.000668 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:43.011587 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:43.011542 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" event={"ID":"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78","Type":"ContainerStarted","Data":"1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0"} Sep 08 07:18:43.836595 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:43.836557 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:43.836764 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:43.836599 2571 patch_prober.go:28] interesting pod/e2e-inf-a-620112-openshift-default-664bd57b66-927l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 08 07:18:43.836764 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:43.836652 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 08 07:18:43.940026 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:43.939992 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:18:44.837200 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:44.837149 2571 patch_prober.go:28] interesting pod/e2e-inf-a-620112-openshift-default-664bd57b66-927l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 08 07:18:44.837626 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:44.837244 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 08 07:18:45.836768 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:45.836732 2571 patch_prober.go:28] interesting pod/e2e-inf-a-620112-openshift-default-664bd57b66-927l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 08 07:18:45.836932 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:45.836789 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 08 07:18:46.836912 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:46.836868 2571 patch_prober.go:28] interesting pod/e2e-inf-a-620112-openshift-default-664bd57b66-927l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 08 07:18:46.837474 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:46.836956 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 08 07:18:47.837010 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:47.836976 2571 patch_prober.go:28] interesting pod/e2e-inf-a-620112-openshift-default-664bd57b66-927l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 08 07:18:47.837432 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:47.837041 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 08 07:18:47.967037 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:47.967002 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:18:48.209455 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.209414 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq"] Sep 08 07:18:48.213688 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.213660 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.216984 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.216965 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-6f5c4f-openshift-default-dockercfg-vfcdp\"" Sep 08 07:18:48.232206 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.232173 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq"] Sep 08 07:18:48.284707 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.284593 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-token\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.284707 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.284677 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-socket\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.284707 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.284710 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istiod-ca-cert\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.285010 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.284739 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-credential-socket\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.285010 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.284884 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-data\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.285010 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.284950 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-certs\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.285171 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.285030 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-podinfo\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.285171 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.285067 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-envoy\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.285171 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.285105 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccclm\" (UniqueName: \"kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-kube-api-access-ccclm\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386006 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.385973 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-data\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386203 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386020 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-certs\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386203 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386054 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-podinfo\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386203 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386077 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-envoy\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386203 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386110 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccclm\" (UniqueName: \"kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-kube-api-access-ccclm\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386203 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386174 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-token\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386238 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-socket\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386270 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istiod-ca-cert\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386300 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-credential-socket\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386437 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-data\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386707 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386684 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-credential-socket\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.386892 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386867 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-certs\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.387084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.386872 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-socket\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.387394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.387371 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istiod-ca-cert\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.388550 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.388520 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-envoy\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.388650 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.388633 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-podinfo\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.394522 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.394491 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-token\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.394811 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.394790 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccclm\" (UniqueName: \"kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-kube-api-access-ccclm\") pod \"e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.528860 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.528771 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:48.828200 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.828173 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq"] Sep 08 07:18:48.829546 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:18:48.829517 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd78c9a_8f85_4d85_8338_8012225a9a1d.slice/crio-bb6582439af94c62700f2009611fffb413c0ccbd79f099c719efcc0f824c8007 WatchSource:0}: Error finding container bb6582439af94c62700f2009611fffb413c0ccbd79f099c719efcc0f824c8007: Status 404 returned error can't find the container with id bb6582439af94c62700f2009611fffb413c0ccbd79f099c719efcc0f824c8007 Sep 08 07:18:48.831960 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.831926 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:48.832063 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.832005 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:48.832063 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.832053 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:48.840920 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.840896 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:48.852868 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.852845 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:48.853561 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:48.853544 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:18:49.058758 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:49.058717 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" event={"ID":"1cd78c9a-8f85-4d85-8338-8012225a9a1d","Type":"ContainerStarted","Data":"27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad"} Sep 08 07:18:49.058974 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:49.058786 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" event={"ID":"1cd78c9a-8f85-4d85-8338-8012225a9a1d","Type":"ContainerStarted","Data":"bb6582439af94c62700f2009611fffb413c0ccbd79f099c719efcc0f824c8007"} Sep 08 07:18:49.077886 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:49.077798 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" podStartSLOduration=1.077777204 podStartE2EDuration="1.077777204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:18:49.076743653 +0000 UTC m=+843.754706792" watchObservedRunningTime="2026-09-08 07:18:49.077777204 +0000 UTC m=+843.755740321" Sep 08 07:18:49.529464 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:49.529426 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:49.534334 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:49.534307 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:50.066591 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:50.066555 2571 generic.go:356] "Generic (PLEG): container finished" podID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerID="f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f" exitCode=0 Sep 08 07:18:50.066978 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:50.066629 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" event={"ID":"ddc173e3-c1f0-48cf-97ff-ea5784652c30","Type":"ContainerDied","Data":"f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f"} Sep 08 07:18:50.067207 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:50.067175 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:50.067357 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:50.067331 2571 scope.go:117] "RemoveContainer" containerID="f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f" Sep 08 07:18:50.068113 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:50.068057 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:18:50.068453 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:50.068427 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:50.068506 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:50.068494 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:50.068542 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:50.068526 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:18:51.074053 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:51.074009 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" event={"ID":"ddc173e3-c1f0-48cf-97ff-ea5784652c30","Type":"ContainerStarted","Data":"c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0"} Sep 08 07:18:51.727818 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:51.727779 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:51.728019 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:51.727996 2571 patch_prober.go:28] interesting pod/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Sep 08 07:18:51.728064 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:51.728048 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Sep 08 07:18:52.727677 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:52.727643 2571 patch_prober.go:28] interesting pod/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Sep 08 07:18:52.728129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:52.727709 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Sep 08 07:18:53.728207 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:53.728164 2571 patch_prober.go:28] interesting pod/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Sep 08 07:18:53.728663 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:53.728242 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Sep 08 07:18:54.727375 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:54.727337 2571 patch_prober.go:28] interesting pod/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Sep 08 07:18:54.727563 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:54.727395 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Sep 08 07:18:55.728162 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:55.728120 2571 patch_prober.go:28] interesting pod/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Sep 08 07:18:55.729098 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:55.728197 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Sep 08 07:18:56.731817 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:56.731788 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:56.732195 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:56.732136 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:18:56.732974 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:18:56.732957 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:19:05.486817 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:05.486783 2571 patch_prober.go:28] interesting pod/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": read tcp 10.134.0.2:55212->10.134.0.47:15021: read: connection reset by peer" start-of-body= Sep 08 07:19:05.487343 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:05.486847 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": read tcp 10.134.0.2:55212->10.134.0.47:15021: read: connection reset by peer" Sep 08 07:19:06.174357 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:06.174322 2571 generic.go:356] "Generic (PLEG): container finished" podID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerID="27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad" exitCode=0 Sep 08 07:19:06.174525 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:06.174401 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" event={"ID":"1cd78c9a-8f85-4d85-8338-8012225a9a1d","Type":"ContainerDied","Data":"27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad"} Sep 08 07:19:06.174859 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:06.174842 2571 scope.go:117] "RemoveContainer" containerID="27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad" Sep 08 07:19:06.175983 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:06.175954 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:19:06.176077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:06.176029 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:19:06.176077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:06.176063 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:19:07.187064 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:07.187023 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" event={"ID":"1cd78c9a-8f85-4d85-8338-8012225a9a1d","Type":"ContainerStarted","Data":"b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7"} Sep 08 07:19:07.529271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:07.529242 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:19:07.529486 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:07.529463 2571 patch_prober.go:28] interesting pod/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Sep 08 07:19:07.529537 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:07.529514 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Sep 08 07:19:08.529386 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:08.529351 2571 patch_prober.go:28] interesting pod/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Sep 08 07:19:08.529792 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:08.529407 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Sep 08 07:19:09.529747 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:09.529714 2571 patch_prober.go:28] interesting pod/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Sep 08 07:19:09.530173 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:09.529772 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Sep 08 07:19:10.529476 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:10.529438 2571 patch_prober.go:28] interesting pod/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Sep 08 07:19:10.529657 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:10.529492 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Sep 08 07:19:11.529923 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:11.529888 2571 patch_prober.go:28] interesting pod/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Sep 08 07:19:11.530362 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:11.529942 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Sep 08 07:19:12.300502 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.300463 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6"] Sep 08 07:19:12.300727 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.300701 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" podUID="2f4aa856-7119-4683-b3b5-ff50382f99ee" containerName="payload-pre-processing" containerID="cri-o://cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f" gracePeriod=30 Sep 08 07:19:12.574151 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.572843 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:19:12.574151 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.574078 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:19:12.584341 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.583862 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:19:12.587892 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.586495 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq"] Sep 08 07:19:12.587892 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.586797 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" containerID="cri-o://c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0" gracePeriod=30 Sep 08 07:19:12.590200 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.590016 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:19:12.652660 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.652595 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f4aa856-7119-4683-b3b5-ff50382f99ee-plugins-config-volume\") pod \"2f4aa856-7119-4683-b3b5-ff50382f99ee\" (UID: \"2f4aa856-7119-4683-b3b5-ff50382f99ee\") " Sep 08 07:19:12.652856 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.652686 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4hc4\" (UniqueName: \"kubernetes.io/projected/2f4aa856-7119-4683-b3b5-ff50382f99ee-kube-api-access-r4hc4\") pod \"2f4aa856-7119-4683-b3b5-ff50382f99ee\" (UID: \"2f4aa856-7119-4683-b3b5-ff50382f99ee\") " Sep 08 07:19:12.653357 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.653064 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4aa856-7119-4683-b3b5-ff50382f99ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f4aa856-7119-4683-b3b5-ff50382f99ee" (UID: "2f4aa856-7119-4683-b3b5-ff50382f99ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:19:12.656136 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.656105 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f4aa856-7119-4683-b3b5-ff50382f99ee-kube-api-access-r4hc4" (OuterVolumeSpecName: "kube-api-access-r4hc4") pod "2f4aa856-7119-4683-b3b5-ff50382f99ee" (UID: "2f4aa856-7119-4683-b3b5-ff50382f99ee"). InnerVolumeSpecName "kube-api-access-r4hc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:19:12.754482 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.754450 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f4aa856-7119-4683-b3b5-ff50382f99ee-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:12.754482 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:12.754476 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4hc4\" (UniqueName: \"kubernetes.io/projected/2f4aa856-7119-4683-b3b5-ff50382f99ee-kube-api-access-r4hc4\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:13.235554 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:13.235522 2571 generic.go:356] "Generic (PLEG): container finished" podID="2f4aa856-7119-4683-b3b5-ff50382f99ee" containerID="cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f" exitCode=0 Sep 08 07:19:13.235747 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:13.235565 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" Sep 08 07:19:13.235747 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:13.235566 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" event={"ID":"2f4aa856-7119-4683-b3b5-ff50382f99ee","Type":"ContainerDied","Data":"cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f"} Sep 08 07:19:13.235747 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:13.235627 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6" event={"ID":"2f4aa856-7119-4683-b3b5-ff50382f99ee","Type":"ContainerDied","Data":"fae7eb52b042964e31ccbf63c935c0fbdc07f7c0757d0e5675d00d235ba7aa39"} Sep 08 07:19:13.235747 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:13.235652 2571 scope.go:117] "RemoveContainer" containerID="cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f" Sep 08 07:19:13.245199 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:13.245180 2571 scope.go:117] "RemoveContainer" containerID="cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f" Sep 08 07:19:13.245496 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:19:13.245478 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f\": container with ID starting with cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f not found: ID does not exist" containerID="cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f" Sep 08 07:19:13.245581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:13.245503 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f"} err="failed to get container status \"cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f\": rpc error: code = NotFound desc = could not find container \"cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f\": container with ID starting with cbd9b758253a891110ae23a7c64c8b99b9546938b6889decdacf70427fae268f not found: ID does not exist" Sep 08 07:19:13.265166 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:13.265143 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6"] Sep 08 07:19:13.273457 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:13.273436 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4a1fb56c-68b9d54c4-5wvn6"] Sep 08 07:19:13.809427 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:13.809392 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f4aa856-7119-4683-b3b5-ff50382f99ee" path="/var/lib/kubelet/pods/2f4aa856-7119-4683-b3b5-ff50382f99ee/volumes" Sep 08 07:19:17.834407 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.834376 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:19:17.909246 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909145 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-certs\") pod \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " Sep 08 07:19:17.909246 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909196 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istiod-ca-cert\") pod \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " Sep 08 07:19:17.909481 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909269 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-envoy\") pod \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " Sep 08 07:19:17.909481 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909296 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-podinfo\") pod \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " Sep 08 07:19:17.909481 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909320 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-credential-socket\") pod \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " Sep 08 07:19:17.909481 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909349 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-socket\") pod \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " Sep 08 07:19:17.909481 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909377 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk9qr\" (UniqueName: \"kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-kube-api-access-kk9qr\") pod \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " Sep 08 07:19:17.909481 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909427 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ddc173e3-c1f0-48cf-97ff-ea5784652c30" (UID: "ddc173e3-c1f0-48cf-97ff-ea5784652c30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:19:17.909481 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909446 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-data\") pod \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " Sep 08 07:19:17.909839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909583 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-token\") pod \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\" (UID: \"ddc173e3-c1f0-48cf-97ff-ea5784652c30\") " Sep 08 07:19:17.909839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909635 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ddc173e3-c1f0-48cf-97ff-ea5784652c30" (UID: "ddc173e3-c1f0-48cf-97ff-ea5784652c30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:19:17.909839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909703 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ddc173e3-c1f0-48cf-97ff-ea5784652c30" (UID: "ddc173e3-c1f0-48cf-97ff-ea5784652c30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:19:17.909839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909716 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-data" (OuterVolumeSpecName: "istio-data") pod "ddc173e3-c1f0-48cf-97ff-ea5784652c30" (UID: "ddc173e3-c1f0-48cf-97ff-ea5784652c30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:19:17.909839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909727 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ddc173e3-c1f0-48cf-97ff-ea5784652c30" (UID: "ddc173e3-c1f0-48cf-97ff-ea5784652c30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:19:17.910097 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909968 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-data\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:17.910097 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.909990 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:17.910097 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.910004 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istiod-ca-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:17.910097 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.910017 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-credential-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:17.910097 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.910032 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-workload-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:17.911507 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.911483 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ddc173e3-c1f0-48cf-97ff-ea5784652c30" (UID: "ddc173e3-c1f0-48cf-97ff-ea5784652c30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 07:19:17.911612 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.911534 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ddc173e3-c1f0-48cf-97ff-ea5784652c30" (UID: "ddc173e3-c1f0-48cf-97ff-ea5784652c30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:19:17.911612 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.911559 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-kube-api-access-kk9qr" (OuterVolumeSpecName: "kube-api-access-kk9qr") pod "ddc173e3-c1f0-48cf-97ff-ea5784652c30" (UID: "ddc173e3-c1f0-48cf-97ff-ea5784652c30"). InnerVolumeSpecName "kube-api-access-kk9qr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:19:17.911612 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:17.911583 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-token" (OuterVolumeSpecName: "istio-token") pod "ddc173e3-c1f0-48cf-97ff-ea5784652c30" (UID: "ddc173e3-c1f0-48cf-97ff-ea5784652c30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:19:18.011230 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.011170 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-token\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:18.011230 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.011203 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-envoy\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:18.011230 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.011235 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddc173e3-c1f0-48cf-97ff-ea5784652c30-istio-podinfo\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:18.011463 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.011244 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk9qr\" (UniqueName: \"kubernetes.io/projected/ddc173e3-c1f0-48cf-97ff-ea5784652c30-kube-api-access-kk9qr\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:18.272065 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.272032 2571 generic.go:356] "Generic (PLEG): container finished" podID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerID="c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0" exitCode=0 Sep 08 07:19:18.272284 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.272089 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" Sep 08 07:19:18.272284 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.272111 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" event={"ID":"ddc173e3-c1f0-48cf-97ff-ea5784652c30","Type":"ContainerDied","Data":"c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0"} Sep 08 07:19:18.272284 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.272186 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq" event={"ID":"ddc173e3-c1f0-48cf-97ff-ea5784652c30","Type":"ContainerDied","Data":"d51081b739faf1a3f495a999819ab9d24b44d293d6c33bf3085c1265ff4e72d2"} Sep 08 07:19:18.272284 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.272208 2571 scope.go:117] "RemoveContainer" containerID="c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0" Sep 08 07:19:18.284029 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.284011 2571 scope.go:117] "RemoveContainer" containerID="f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f" Sep 08 07:19:18.292828 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.292810 2571 scope.go:117] "RemoveContainer" containerID="c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0" Sep 08 07:19:18.293096 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:19:18.293080 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0\": container with ID starting with c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0 not found: ID does not exist" containerID="c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0" Sep 08 07:19:18.293143 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.293103 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0"} err="failed to get container status \"c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0\": rpc error: code = NotFound desc = could not find container \"c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0\": container with ID starting with c2d40a53f9464aa4f2370685c578d2ed442feb1e939590b2cc246ac953b84da0 not found: ID does not exist" Sep 08 07:19:18.293143 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.293119 2571 scope.go:117] "RemoveContainer" containerID="f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f" Sep 08 07:19:18.293474 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:19:18.293451 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f\": container with ID starting with f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f not found: ID does not exist" containerID="f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f" Sep 08 07:19:18.293532 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.293473 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f"} err="failed to get container status \"f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f\": rpc error: code = NotFound desc = could not find container \"f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f\": container with ID starting with f4127afdbd2c3f5e8d4eab8a56c07f368f517c98e175e275037aa974d427be3f not found: ID does not exist" Sep 08 07:19:18.297253 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.297207 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq"] Sep 08 07:19:18.300584 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:18.300565 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4a1fb56c-openshift-default-c87f94fc6-8dvnq"] Sep 08 07:19:19.809454 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:19.809419 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" path="/var/lib/kubelet/pods/ddc173e3-c1f0-48cf-97ff-ea5784652c30/volumes" Sep 08 07:19:41.256442 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.256406 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn"] Sep 08 07:19:41.256845 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.256805 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" Sep 08 07:19:41.256845 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.256816 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" Sep 08 07:19:41.256845 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.256825 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4aa856-7119-4683-b3b5-ff50382f99ee" containerName="payload-pre-processing" Sep 08 07:19:41.256845 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.256830 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4aa856-7119-4683-b3b5-ff50382f99ee" containerName="payload-pre-processing" Sep 08 07:19:41.256845 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.256841 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" Sep 08 07:19:41.256845 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.256846 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" Sep 08 07:19:41.257034 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.256910 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f4aa856-7119-4683-b3b5-ff50382f99ee" containerName="payload-pre-processing" Sep 08 07:19:41.257034 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.256917 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" Sep 08 07:19:41.257034 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.256923 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddc173e3-c1f0-48cf-97ff-ea5784652c30" containerName="istio-proxy" Sep 08 07:19:41.260266 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.260247 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" Sep 08 07:19:41.263325 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.263304 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-824nh\"" Sep 08 07:19:41.263872 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.263856 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 07:19:41.270945 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.270919 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn"] Sep 08 07:19:41.341278 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.341239 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52b32f02-88d6-473f-85c6-bcd836be79a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn\" (UID: \"52b32f02-88d6-473f-85c6-bcd836be79a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" Sep 08 07:19:41.442622 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.442586 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52b32f02-88d6-473f-85c6-bcd836be79a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn\" (UID: \"52b32f02-88d6-473f-85c6-bcd836be79a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" Sep 08 07:19:41.443187 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.443165 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52b32f02-88d6-473f-85c6-bcd836be79a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn\" (UID: \"52b32f02-88d6-473f-85c6-bcd836be79a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" Sep 08 07:19:41.571470 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.571369 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" Sep 08 07:19:41.850406 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:41.850358 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn"] Sep 08 07:19:41.852664 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:19:41.852640 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-6564936936dd4bb333615cdceb2316d36c44977869ce4670985f0348aa72831c WatchSource:0}: Error finding container 6564936936dd4bb333615cdceb2316d36c44977869ce4670985f0348aa72831c: Status 404 returned error can't find the container with id 6564936936dd4bb333615cdceb2316d36c44977869ce4670985f0348aa72831c Sep 08 07:19:42.407385 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:42.407349 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" event={"ID":"52b32f02-88d6-473f-85c6-bcd836be79a0","Type":"ContainerStarted","Data":"6564936936dd4bb333615cdceb2316d36c44977869ce4670985f0348aa72831c"} Sep 08 07:19:45.425799 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:45.425761 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" event={"ID":"52b32f02-88d6-473f-85c6-bcd836be79a0","Type":"ContainerStarted","Data":"18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a"} Sep 08 07:19:45.448931 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:45.448878 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" podStartSLOduration=1.35094914 podStartE2EDuration="4.448862144s" totalImagesPullingTime="3.097913004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:19:45.44673952 +0000 UTC m=+900.124702663" watchObservedRunningTime="2026-09-08 07:19:45.448862144 +0000 UTC m=+900.126825259" Sep 08 07:19:46.432826 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:46.432783 2571 generic.go:356] "Generic (PLEG): container finished" podID="52b32f02-88d6-473f-85c6-bcd836be79a0" containerID="18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a" exitCode=0 Sep 08 07:19:46.433202 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:46.432829 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" event={"ID":"52b32f02-88d6-473f-85c6-bcd836be79a0","Type":"ContainerDied","Data":"18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a"} Sep 08 07:19:47.568259 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:47.568234 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" Sep 08 07:19:47.603425 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:47.600462 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52b32f02-88d6-473f-85c6-bcd836be79a0-maas-api-service-ca\") pod \"52b32f02-88d6-473f-85c6-bcd836be79a0\" (UID: \"52b32f02-88d6-473f-85c6-bcd836be79a0\") " Sep 08 07:19:47.603425 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:47.601168 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52b32f02-88d6-473f-85c6-bcd836be79a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52b32f02-88d6-473f-85c6-bcd836be79a0" (UID: "52b32f02-88d6-473f-85c6-bcd836be79a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:19:47.702101 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:47.702070 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52b32f02-88d6-473f-85c6-bcd836be79a0-maas-api-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:19:48.228984 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:19:48.228939 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:19:48.446090 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:48.446052 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" event={"ID":"52b32f02-88d6-473f-85c6-bcd836be79a0","Type":"ContainerDied","Data":"6564936936dd4bb333615cdceb2316d36c44977869ce4670985f0348aa72831c"} Sep 08 07:19:48.446301 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:48.446112 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6564936936dd4bb333615cdceb2316d36c44977869ce4670985f0348aa72831c" Sep 08 07:19:48.446301 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:48.446064 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn" Sep 08 07:19:48.459701 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:19:48.459671 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:19:57.238374 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.238342 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf"] Sep 08 07:19:57.238776 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.238743 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52b32f02-88d6-473f-85c6-bcd836be79a0" containerName="revoke-keys" Sep 08 07:19:57.238776 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.238753 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="52b32f02-88d6-473f-85c6-bcd836be79a0" containerName="revoke-keys" Sep 08 07:19:57.238862 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.238822 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="52b32f02-88d6-473f-85c6-bcd836be79a0" containerName="revoke-keys" Sep 08 07:19:57.241049 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.241033 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.243561 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.243539 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-638d2d\"/\"openshift-service-ca.crt\"" Sep 08 07:19:57.243684 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.243539 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-638d2d\"/\"autores-model-638d2d-kserve-self-signed-certs\"" Sep 08 07:19:57.244807 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.244789 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-638d2d\"/\"default-dockercfg-2w9zc\"" Sep 08 07:19:57.244913 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.244821 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-638d2d\"/\"kube-root-ca.crt\"" Sep 08 07:19:57.254145 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.254115 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf"] Sep 08 07:19:57.305724 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.305688 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-model-cache\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.305899 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.305732 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tls-certs\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.305899 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.305756 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-home\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.305899 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.305802 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr9tz\" (UniqueName: \"kubernetes.io/projected/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-kube-api-access-mr9tz\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.305899 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.305851 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tmp-dir\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.305899 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.305889 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-dshm\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.407124 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.407087 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr9tz\" (UniqueName: \"kubernetes.io/projected/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-kube-api-access-mr9tz\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.407353 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.407153 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tmp-dir\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.407353 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.407205 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-dshm\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.407353 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.407308 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-model-cache\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.407353 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.407338 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tls-certs\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.407585 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.407362 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-home\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.407585 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.407557 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tmp-dir\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.407726 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.407705 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-home\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.407791 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.407721 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-model-cache\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.409543 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.409525 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-dshm\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.409638 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.409620 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tls-certs\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.421166 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.421142 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr9tz\" (UniqueName: \"kubernetes.io/projected/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-kube-api-access-mr9tz\") pod \"autores-model-638d2d-kserve-598f845b4c-x2wrf\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.552494 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.552399 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:19:57.828434 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:57.828406 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf"] Sep 08 07:19:57.830016 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:19:57.829991 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91594dc2_1fa5_4fc3_afe2_e5dd4b8b66e8.slice/crio-fb38c3b5a0ffa91f42308f2e1c5c9ad37c9d5d72709bd2432b5fce9d5623840d WatchSource:0}: Error finding container fb38c3b5a0ffa91f42308f2e1c5c9ad37c9d5d72709bd2432b5fce9d5623840d: Status 404 returned error can't find the container with id fb38c3b5a0ffa91f42308f2e1c5c9ad37c9d5d72709bd2432b5fce9d5623840d Sep 08 07:19:58.507882 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:58.507844 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" event={"ID":"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8","Type":"ContainerStarted","Data":"b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697"} Sep 08 07:19:58.508275 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:58.507893 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" event={"ID":"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8","Type":"ContainerStarted","Data":"fb38c3b5a0ffa91f42308f2e1c5c9ad37c9d5d72709bd2432b5fce9d5623840d"} Sep 08 07:19:58.522075 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:19:58.522046 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:19:58.534719 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:58.534676 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" podStartSLOduration=1.5346629269999998 podStartE2EDuration="1.534662927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:19:58.533603734 +0000 UTC m=+913.211566850" watchObservedRunningTime="2026-09-08 07:19:58.534662927 +0000 UTC m=+913.212626044" Sep 08 07:19:58.539793 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:58.539770 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn"] Sep 08 07:19:58.545871 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:58.545849 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3da4827-prbsn"] Sep 08 07:19:59.816248 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:19:59.816183 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52b32f02-88d6-473f-85c6-bcd836be79a0" path="/var/lib/kubelet/pods/52b32f02-88d6-473f-85c6-bcd836be79a0/volumes" Sep 08 07:20:03.190472 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:20:03.190426 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:20:03.706004 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.705970 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl"] Sep 08 07:20:03.709100 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.709084 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:03.711471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.711448 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-576ac2af\"" Sep 08 07:20:03.711576 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.711548 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-576ac2af-dockercfg-46zcc\"" Sep 08 07:20:03.725710 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.725676 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl"] Sep 08 07:20:03.879013 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.878984 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b2d960d-f97c-48b7-839f-4b295376e8dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl\" (UID: \"8b2d960d-f97c-48b7-839f-4b295376e8dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:03.879183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.879095 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8tqk\" (UniqueName: \"kubernetes.io/projected/8b2d960d-f97c-48b7-839f-4b295376e8dd-kube-api-access-j8tqk\") pod \"payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl\" (UID: \"8b2d960d-f97c-48b7-839f-4b295376e8dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:03.980647 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.980553 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8tqk\" (UniqueName: \"kubernetes.io/projected/8b2d960d-f97c-48b7-839f-4b295376e8dd-kube-api-access-j8tqk\") pod \"payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl\" (UID: \"8b2d960d-f97c-48b7-839f-4b295376e8dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:03.980824 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.980675 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b2d960d-f97c-48b7-839f-4b295376e8dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl\" (UID: \"8b2d960d-f97c-48b7-839f-4b295376e8dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:03.981488 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.981468 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b2d960d-f97c-48b7-839f-4b295376e8dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl\" (UID: \"8b2d960d-f97c-48b7-839f-4b295376e8dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:03.993245 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:03.993201 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8tqk\" (UniqueName: \"kubernetes.io/projected/8b2d960d-f97c-48b7-839f-4b295376e8dd-kube-api-access-j8tqk\") pod \"payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl\" (UID: \"8b2d960d-f97c-48b7-839f-4b295376e8dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:04.019515 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:04.019486 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:04.306589 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:04.306565 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl"] Sep 08 07:20:04.309186 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:20:04.309154 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b2d960d_f97c_48b7_839f_4b295376e8dd.slice/crio-9f13cc8a77159d1fc75b4df5a7f7a82128866fd02c41fbdc8c3f014e7d41d81a WatchSource:0}: Error finding container 9f13cc8a77159d1fc75b4df5a7f7a82128866fd02c41fbdc8c3f014e7d41d81a: Status 404 returned error can't find the container with id 9f13cc8a77159d1fc75b4df5a7f7a82128866fd02c41fbdc8c3f014e7d41d81a Sep 08 07:20:04.546273 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:04.546237 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" event={"ID":"8b2d960d-f97c-48b7-839f-4b295376e8dd","Type":"ContainerStarted","Data":"47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03"} Sep 08 07:20:04.546438 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:04.546289 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" event={"ID":"8b2d960d-f97c-48b7-839f-4b295376e8dd","Type":"ContainerStarted","Data":"9f13cc8a77159d1fc75b4df5a7f7a82128866fd02c41fbdc8c3f014e7d41d81a"} Sep 08 07:20:04.546438 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:04.546371 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:04.569488 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:04.569393 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" podStartSLOduration=1.569378211 podStartE2EDuration="1.569378211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:20:04.567820752 +0000 UTC m=+919.245783869" watchObservedRunningTime="2026-09-08 07:20:04.569378211 +0000 UTC m=+919.247341327" Sep 08 07:20:04.976738 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:20:04.976693 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:20:04.976950 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:20:04.976750 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:20:07.553596 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:07.553558 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:20:07.554020 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:07.553641 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:20:07.566295 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:07.566271 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:20:08.531537 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:20:08.531506 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:20:08.579995 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:08.579933 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:20:15.554410 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:15.554379 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:18.247291 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:20:18.247257 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:20:18.541488 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:20:18.541402 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:20:28.588827 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:20:28.588785 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:20:31.814957 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:31.814926 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2"] Sep 08 07:20:31.819808 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:31.819787 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" Sep 08 07:20:31.822731 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:31.822700 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-824nh\"" Sep 08 07:20:31.824291 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:31.824267 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 07:20:31.826385 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:31.826359 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2"] Sep 08 07:20:31.955160 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:31.955123 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c84ebb-e09d-417b-83fd-bc87ba90a49a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-638d2d-hmpq2\" (UID: \"b4c84ebb-e09d-417b-83fd-bc87ba90a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" Sep 08 07:20:32.056410 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:32.056370 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c84ebb-e09d-417b-83fd-bc87ba90a49a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-638d2d-hmpq2\" (UID: \"b4c84ebb-e09d-417b-83fd-bc87ba90a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" Sep 08 07:20:32.056973 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:32.056953 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c84ebb-e09d-417b-83fd-bc87ba90a49a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-638d2d-hmpq2\" (UID: \"b4c84ebb-e09d-417b-83fd-bc87ba90a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" Sep 08 07:20:32.131428 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:32.131347 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" Sep 08 07:20:32.403914 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:32.403883 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2"] Sep 08 07:20:32.404578 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:20:32.404549 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c84ebb_e09d_417b_83fd_bc87ba90a49a.slice/crio-08866f20d2106e3f6bd86af16d708edc205ee59cc7f08db9f7e3b5c9f3892749 WatchSource:0}: Error finding container 08866f20d2106e3f6bd86af16d708edc205ee59cc7f08db9f7e3b5c9f3892749: Status 404 returned error can't find the container with id 08866f20d2106e3f6bd86af16d708edc205ee59cc7f08db9f7e3b5c9f3892749 Sep 08 07:20:32.706783 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:32.706749 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" event={"ID":"b4c84ebb-e09d-417b-83fd-bc87ba90a49a","Type":"ContainerStarted","Data":"d7c5280d8b47d0524d4113a5bb1d41fcc6a34549bfa14cfaeb1ba1625e904b3a"} Sep 08 07:20:32.706950 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:32.706805 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" event={"ID":"b4c84ebb-e09d-417b-83fd-bc87ba90a49a","Type":"ContainerStarted","Data":"08866f20d2106e3f6bd86af16d708edc205ee59cc7f08db9f7e3b5c9f3892749"} Sep 08 07:20:32.723526 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:32.723475 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" podStartSLOduration=1.7234602909999999 podStartE2EDuration="1.723460291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:20:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:20:32.721566611 +0000 UTC m=+947.399529747" watchObservedRunningTime="2026-09-08 07:20:32.723460291 +0000 UTC m=+947.401423403" Sep 08 07:20:33.202858 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:20:33.202822 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:20:33.222352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.222322 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl"] Sep 08 07:20:33.222582 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.222557 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" podUID="8b2d960d-f97c-48b7-839f-4b295376e8dd" containerName="payload-pre-processing" containerID="cri-o://47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03" gracePeriod=30 Sep 08 07:20:33.471953 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.471925 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:33.571241 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.570547 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8tqk\" (UniqueName: \"kubernetes.io/projected/8b2d960d-f97c-48b7-839f-4b295376e8dd-kube-api-access-j8tqk\") pod \"8b2d960d-f97c-48b7-839f-4b295376e8dd\" (UID: \"8b2d960d-f97c-48b7-839f-4b295376e8dd\") " Sep 08 07:20:33.571241 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.570715 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b2d960d-f97c-48b7-839f-4b295376e8dd-plugins-config-volume\") pod \"8b2d960d-f97c-48b7-839f-4b295376e8dd\" (UID: \"8b2d960d-f97c-48b7-839f-4b295376e8dd\") " Sep 08 07:20:33.571576 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.571358 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b2d960d-f97c-48b7-839f-4b295376e8dd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8b2d960d-f97c-48b7-839f-4b295376e8dd" (UID: "8b2d960d-f97c-48b7-839f-4b295376e8dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:20:33.574136 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.574098 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b2d960d-f97c-48b7-839f-4b295376e8dd-kube-api-access-j8tqk" (OuterVolumeSpecName: "kube-api-access-j8tqk") pod "8b2d960d-f97c-48b7-839f-4b295376e8dd" (UID: "8b2d960d-f97c-48b7-839f-4b295376e8dd"). InnerVolumeSpecName "kube-api-access-j8tqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:20:33.672246 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.672153 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b2d960d-f97c-48b7-839f-4b295376e8dd-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:20:33.672246 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.672182 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8tqk\" (UniqueName: \"kubernetes.io/projected/8b2d960d-f97c-48b7-839f-4b295376e8dd-kube-api-access-j8tqk\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:20:33.716699 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.716668 2571 generic.go:356] "Generic (PLEG): container finished" podID="b4c84ebb-e09d-417b-83fd-bc87ba90a49a" containerID="d7c5280d8b47d0524d4113a5bb1d41fcc6a34549bfa14cfaeb1ba1625e904b3a" exitCode=0 Sep 08 07:20:33.716888 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.716773 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" event={"ID":"b4c84ebb-e09d-417b-83fd-bc87ba90a49a","Type":"ContainerDied","Data":"d7c5280d8b47d0524d4113a5bb1d41fcc6a34549bfa14cfaeb1ba1625e904b3a"} Sep 08 07:20:33.725820 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.725791 2571 generic.go:356] "Generic (PLEG): container finished" podID="8b2d960d-f97c-48b7-839f-4b295376e8dd" containerID="47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03" exitCode=0 Sep 08 07:20:33.725992 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.725848 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" Sep 08 07:20:33.725992 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.725876 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" event={"ID":"8b2d960d-f97c-48b7-839f-4b295376e8dd","Type":"ContainerDied","Data":"47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03"} Sep 08 07:20:33.725992 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.725949 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl" event={"ID":"8b2d960d-f97c-48b7-839f-4b295376e8dd","Type":"ContainerDied","Data":"9f13cc8a77159d1fc75b4df5a7f7a82128866fd02c41fbdc8c3f014e7d41d81a"} Sep 08 07:20:33.725992 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.725972 2571 scope.go:117] "RemoveContainer" containerID="47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03" Sep 08 07:20:33.739447 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.739427 2571 scope.go:117] "RemoveContainer" containerID="47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03" Sep 08 07:20:33.739797 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:20:33.739776 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03\": container with ID starting with 47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03 not found: ID does not exist" containerID="47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03" Sep 08 07:20:33.739863 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.739805 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03"} err="failed to get container status \"47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03\": rpc error: code = NotFound desc = could not find container \"47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03\": container with ID starting with 47ff116bae097d120f2b00ef13ecb66df41475546536d574b3e15e6b4075af03 not found: ID does not exist" Sep 08 07:20:33.755387 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.755358 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl"] Sep 08 07:20:33.759409 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.759382 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-576ac2af-779b4c8678-bx9sl"] Sep 08 07:20:33.809667 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:33.809634 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b2d960d-f97c-48b7-839f-4b295376e8dd" path="/var/lib/kubelet/pods/8b2d960d-f97c-48b7-839f-4b295376e8dd/volumes" Sep 08 07:20:34.860062 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:34.860037 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" Sep 08 07:20:34.986142 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:34.986062 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c84ebb-e09d-417b-83fd-bc87ba90a49a-maas-api-service-ca\") pod \"b4c84ebb-e09d-417b-83fd-bc87ba90a49a\" (UID: \"b4c84ebb-e09d-417b-83fd-bc87ba90a49a\") " Sep 08 07:20:34.986498 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:34.986473 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4c84ebb-e09d-417b-83fd-bc87ba90a49a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b4c84ebb-e09d-417b-83fd-bc87ba90a49a" (UID: "b4c84ebb-e09d-417b-83fd-bc87ba90a49a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:20:35.087792 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:35.087753 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c84ebb-e09d-417b-83fd-bc87ba90a49a-maas-api-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:20:35.739930 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:35.739896 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" Sep 08 07:20:35.739930 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:35.739915 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2" event={"ID":"b4c84ebb-e09d-417b-83fd-bc87ba90a49a","Type":"ContainerDied","Data":"08866f20d2106e3f6bd86af16d708edc205ee59cc7f08db9f7e3b5c9f3892749"} Sep 08 07:20:35.740150 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:35.739962 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08866f20d2106e3f6bd86af16d708edc205ee59cc7f08db9f7e3b5c9f3892749" Sep 08 07:20:38.641489 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:20:38.641452 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b32f02_88d6_473f_85c6_bcd836be79a0.slice/crio-18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a.scope\": RecentStats: unable to find data in memory cache]" Sep 08 07:20:39.524650 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.524614 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd"] Sep 08 07:20:39.525045 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.525032 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b2d960d-f97c-48b7-839f-4b295376e8dd" containerName="payload-pre-processing" Sep 08 07:20:39.525091 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.525046 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b2d960d-f97c-48b7-839f-4b295376e8dd" containerName="payload-pre-processing" Sep 08 07:20:39.525091 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.525060 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4c84ebb-e09d-417b-83fd-bc87ba90a49a" containerName="revoke-keys" Sep 08 07:20:39.525091 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.525066 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4c84ebb-e09d-417b-83fd-bc87ba90a49a" containerName="revoke-keys" Sep 08 07:20:39.525187 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.525135 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b2d960d-f97c-48b7-839f-4b295376e8dd" containerName="payload-pre-processing" Sep 08 07:20:39.525187 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.525145 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4c84ebb-e09d-417b-83fd-bc87ba90a49a" containerName="revoke-keys" Sep 08 07:20:39.527178 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.527162 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:20:39.533811 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.533541 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-da54544b-dockercfg-vg9sv\"" Sep 08 07:20:39.533811 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.533554 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-da54544b\"" Sep 08 07:20:39.539056 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.539030 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd"] Sep 08 07:20:39.631594 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.631557 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd\" (UID: \"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:20:39.631594 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.631602 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-468m6\" (UniqueName: \"kubernetes.io/projected/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-kube-api-access-468m6\") pod \"payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd\" (UID: \"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:20:39.732550 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.732512 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd\" (UID: \"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:20:39.732997 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.732561 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-468m6\" (UniqueName: \"kubernetes.io/projected/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-kube-api-access-468m6\") pod \"payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd\" (UID: \"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:20:39.733526 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.733501 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd\" (UID: \"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:20:39.740796 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.740770 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-468m6\" (UniqueName: \"kubernetes.io/projected/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-kube-api-access-468m6\") pod \"payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd\" (UID: \"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:20:39.837585 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:39.837497 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:20:40.110065 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:40.110036 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd"] Sep 08 07:20:40.110824 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:20:40.110802 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a5a069_dcdc_496c_9d85_20e6fc7db1ec.slice/crio-3e69cb5e18d7038d5f3e65b37ccc0fc6d6624b2c8409b031a7300271a6aa29b4 WatchSource:0}: Error finding container 3e69cb5e18d7038d5f3e65b37ccc0fc6d6624b2c8409b031a7300271a6aa29b4: Status 404 returned error can't find the container with id 3e69cb5e18d7038d5f3e65b37ccc0fc6d6624b2c8409b031a7300271a6aa29b4 Sep 08 07:20:40.769562 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:40.769528 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" event={"ID":"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec","Type":"ContainerStarted","Data":"23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a"} Sep 08 07:20:40.769922 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:40.769578 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" event={"ID":"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec","Type":"ContainerStarted","Data":"3e69cb5e18d7038d5f3e65b37ccc0fc6d6624b2c8409b031a7300271a6aa29b4"} Sep 08 07:20:40.769922 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:40.769662 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:20:40.786547 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:40.786506 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" podStartSLOduration=1.786492418 podStartE2EDuration="1.786492418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:20:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:20:40.78584399 +0000 UTC m=+955.463807110" watchObservedRunningTime="2026-09-08 07:20:40.786492418 +0000 UTC m=+955.464455533" Sep 08 07:20:42.608466 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:42.608417 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km"] Sep 08 07:20:42.608945 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:42.608657 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" podUID="fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30" containerName="payload-pre-processing" containerID="cri-o://b8f83c81141ba8993a329c9921bc2bdd2e5f3edc2c343d5328eff6d04c894dcb" gracePeriod=30 Sep 08 07:20:42.779485 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:42.779455 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2"] Sep 08 07:20:42.792815 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:42.792775 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-638d2d-hmpq2"] Sep 08 07:20:42.795537 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:42.795506 2571 generic.go:356] "Generic (PLEG): container finished" podID="fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30" containerID="b8f83c81141ba8993a329c9921bc2bdd2e5f3edc2c343d5328eff6d04c894dcb" exitCode=0 Sep 08 07:20:42.795657 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:42.795587 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" event={"ID":"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30","Type":"ContainerDied","Data":"b8f83c81141ba8993a329c9921bc2bdd2e5f3edc2c343d5328eff6d04c894dcb"} Sep 08 07:20:42.873057 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:42.872982 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:20:43.065277 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.065242 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnbwp\" (UniqueName: \"kubernetes.io/projected/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-kube-api-access-xnbwp\") pod \"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30\" (UID: \"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30\") " Sep 08 07:20:43.065466 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.065358 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-plugins-config-volume\") pod \"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30\" (UID: \"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30\") " Sep 08 07:20:43.065706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.065684 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30" (UID: "fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:20:43.067270 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.067244 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-kube-api-access-xnbwp" (OuterVolumeSpecName: "kube-api-access-xnbwp") pod "fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30" (UID: "fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30"). InnerVolumeSpecName "kube-api-access-xnbwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:20:43.166978 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.166895 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:20:43.166978 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.166922 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnbwp\" (UniqueName: \"kubernetes.io/projected/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30-kube-api-access-xnbwp\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:20:43.802259 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.802206 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" Sep 08 07:20:43.802663 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.802253 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km" event={"ID":"fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30","Type":"ContainerDied","Data":"89e4339720f2a84d64ee745bb0898526d9db45bbfa9218007d78e607c7984a1b"} Sep 08 07:20:43.802663 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.802317 2571 scope.go:117] "RemoveContainer" containerID="b8f83c81141ba8993a329c9921bc2bdd2e5f3edc2c343d5328eff6d04c894dcb" Sep 08 07:20:43.809049 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.809019 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4c84ebb-e09d-417b-83fd-bc87ba90a49a" path="/var/lib/kubelet/pods/b4c84ebb-e09d-417b-83fd-bc87ba90a49a/volumes" Sep 08 07:20:43.827557 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.827531 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km"] Sep 08 07:20:43.829895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:43.829875 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-638d2d-cf79ccfd6-xf9km"] Sep 08 07:20:45.809423 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:45.809398 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30" path="/var/lib/kubelet/pods/fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30/volumes" Sep 08 07:20:48.099177 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:48.099142 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf"] Sep 08 07:20:51.777262 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:20:51.777200 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:21:02.506539 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.506503 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5"] Sep 08 07:21:02.506918 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.506891 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30" containerName="payload-pre-processing" Sep 08 07:21:02.506918 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.506902 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30" containerName="payload-pre-processing" Sep 08 07:21:02.506993 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.506984 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd0f2ee4-2b38-4ba8-b30e-d098ca8d2a30" containerName="payload-pre-processing" Sep 08 07:21:02.511604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.511585 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5" Sep 08 07:21:02.515200 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.515169 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 07:21:02.516355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.516332 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-824nh\"" Sep 08 07:21:02.517304 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.517276 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5"] Sep 08 07:21:02.555909 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.555875 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e88561-63a4-4ad3-85e7-dd214f290a8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-da54544b-9hdr5\" (UID: \"92e88561-63a4-4ad3-85e7-dd214f290a8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5" Sep 08 07:21:02.657495 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.657461 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e88561-63a4-4ad3-85e7-dd214f290a8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-da54544b-9hdr5\" (UID: \"92e88561-63a4-4ad3-85e7-dd214f290a8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5" Sep 08 07:21:02.658038 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.658018 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e88561-63a4-4ad3-85e7-dd214f290a8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-da54544b-9hdr5\" (UID: \"92e88561-63a4-4ad3-85e7-dd214f290a8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5" Sep 08 07:21:02.824355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:02.824262 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5" Sep 08 07:21:03.093521 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.093487 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5"] Sep 08 07:21:03.094166 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:21:03.094140 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e88561_63a4_4ad3_85e7_dd214f290a8a.slice/crio-bfb601c77f36e5ec42893c3fd817dcc0a31d35b656da9bbebcf58e40fb563560 WatchSource:0}: Error finding container bfb601c77f36e5ec42893c3fd817dcc0a31d35b656da9bbebcf58e40fb563560: Status 404 returned error can't find the container with id bfb601c77f36e5ec42893c3fd817dcc0a31d35b656da9bbebcf58e40fb563560 Sep 08 07:21:03.119855 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.119794 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" podUID="91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" containerName="main" containerID="cri-o://b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697" gracePeriod=45 Sep 08 07:21:03.357717 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.357694 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:21:03.466167 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466135 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr9tz\" (UniqueName: \"kubernetes.io/projected/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-kube-api-access-mr9tz\") pod \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " Sep 08 07:21:03.466329 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466181 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-dshm\") pod \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " Sep 08 07:21:03.466329 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466275 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-home\") pod \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " Sep 08 07:21:03.466329 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466322 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-model-cache\") pod \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " Sep 08 07:21:03.466482 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466351 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tls-certs\") pod \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " Sep 08 07:21:03.466482 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466390 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tmp-dir\") pod \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\" (UID: \"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8\") " Sep 08 07:21:03.466594 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466560 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-home" (OuterVolumeSpecName: "home") pod "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" (UID: "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:21:03.466660 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466581 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-model-cache" (OuterVolumeSpecName: "model-cache") pod "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" (UID: "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:21:03.466756 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466724 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" (UID: "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:21:03.466756 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466745 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-home\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:21:03.467083 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.466776 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-model-cache\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:21:03.468448 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.468429 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-dshm" (OuterVolumeSpecName: "dshm") pod "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" (UID: "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:21:03.468714 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.468691 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" (UID: "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:21:03.468714 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.468708 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-kube-api-access-mr9tz" (OuterVolumeSpecName: "kube-api-access-mr9tz") pod "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" (UID: "91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8"). InnerVolumeSpecName "kube-api-access-mr9tz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:21:03.567306 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.567268 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tls-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:21:03.567306 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.567297 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-tmp-dir\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:21:03.567306 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.567306 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr9tz\" (UniqueName: \"kubernetes.io/projected/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-kube-api-access-mr9tz\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:21:03.567306 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.567315 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8-dshm\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:21:03.916449 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.916414 2571 generic.go:356] "Generic (PLEG): container finished" podID="92e88561-63a4-4ad3-85e7-dd214f290a8a" containerID="2c49922e4d131cc7efca9ef945b2fb691f04767604561222e3aca4082d8aeb78" exitCode=0 Sep 08 07:21:03.916634 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.916493 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5" event={"ID":"92e88561-63a4-4ad3-85e7-dd214f290a8a","Type":"ContainerDied","Data":"2c49922e4d131cc7efca9ef945b2fb691f04767604561222e3aca4082d8aeb78"} Sep 08 07:21:03.916634 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.916548 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5" event={"ID":"92e88561-63a4-4ad3-85e7-dd214f290a8a","Type":"ContainerStarted","Data":"bfb601c77f36e5ec42893c3fd817dcc0a31d35b656da9bbebcf58e40fb563560"} Sep 08 07:21:03.917821 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.917795 2571 generic.go:356] "Generic (PLEG): container finished" podID="91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" containerID="b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697" exitCode=0 Sep 08 07:21:03.917938 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.917835 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" Sep 08 07:21:03.917938 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.917866 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" event={"ID":"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8","Type":"ContainerDied","Data":"b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697"} Sep 08 07:21:03.917938 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.917901 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf" event={"ID":"91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8","Type":"ContainerDied","Data":"fb38c3b5a0ffa91f42308f2e1c5c9ad37c9d5d72709bd2432b5fce9d5623840d"} Sep 08 07:21:03.918702 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.917922 2571 scope.go:117] "RemoveContainer" containerID="b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697" Sep 08 07:21:03.927460 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.927435 2571 scope.go:117] "RemoveContainer" containerID="b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697" Sep 08 07:21:03.927742 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:21:03.927724 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697\": container with ID starting with b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697 not found: ID does not exist" containerID="b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697" Sep 08 07:21:03.927792 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.927752 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697"} err="failed to get container status \"b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697\": rpc error: code = NotFound desc = could not find container \"b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697\": container with ID starting with b46bcd52a522613a9508169fa1f552ab9ca6066728233c85ef3fd46045b04697 not found: ID does not exist" Sep 08 07:21:03.945467 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.945435 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf"] Sep 08 07:21:03.951192 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:03.951156 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-638d2d/autores-model-638d2d-kserve-598f845b4c-x2wrf"] Sep 08 07:21:05.062806 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:05.062782 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5" Sep 08 07:21:05.183071 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:05.182987 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e88561-63a4-4ad3-85e7-dd214f290a8a-maas-api-service-ca\") pod \"92e88561-63a4-4ad3-85e7-dd214f290a8a\" (UID: \"92e88561-63a4-4ad3-85e7-dd214f290a8a\") " Sep 08 07:21:05.183335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:05.183314 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92e88561-63a4-4ad3-85e7-dd214f290a8a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "92e88561-63a4-4ad3-85e7-dd214f290a8a" (UID: "92e88561-63a4-4ad3-85e7-dd214f290a8a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:21:05.284060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:05.284029 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e88561-63a4-4ad3-85e7-dd214f290a8a-maas-api-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:21:05.810380 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:05.810345 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" path="/var/lib/kubelet/pods/91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8/volumes" Sep 08 07:21:05.932333 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:05.932296 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5" Sep 08 07:21:05.932501 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:05.932343 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5" event={"ID":"92e88561-63a4-4ad3-85e7-dd214f290a8a","Type":"ContainerDied","Data":"bfb601c77f36e5ec42893c3fd817dcc0a31d35b656da9bbebcf58e40fb563560"} Sep 08 07:21:05.932501 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:05.932393 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfb601c77f36e5ec42893c3fd817dcc0a31d35b656da9bbebcf58e40fb563560" Sep 08 07:21:09.639948 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.639747 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd"] Sep 08 07:21:09.640401 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.640024 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" podUID="a7a5a069-dcdc-496c-9d85-20e6fc7db1ec" containerName="payload-pre-processing" containerID="cri-o://23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a" gracePeriod=30 Sep 08 07:21:09.796004 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.795965 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5"] Sep 08 07:21:09.800454 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.800429 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-da54544b-9hdr5"] Sep 08 07:21:09.810745 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.810717 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92e88561-63a4-4ad3-85e7-dd214f290a8a" path="/var/lib/kubelet/pods/92e88561-63a4-4ad3-85e7-dd214f290a8a/volumes" Sep 08 07:21:09.928117 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.928085 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:21:09.961149 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.961115 2571 generic.go:356] "Generic (PLEG): container finished" podID="a7a5a069-dcdc-496c-9d85-20e6fc7db1ec" containerID="23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a" exitCode=0 Sep 08 07:21:09.961349 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.961158 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" Sep 08 07:21:09.961349 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.961162 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" event={"ID":"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec","Type":"ContainerDied","Data":"23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a"} Sep 08 07:21:09.961349 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.961249 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd" event={"ID":"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec","Type":"ContainerDied","Data":"3e69cb5e18d7038d5f3e65b37ccc0fc6d6624b2c8409b031a7300271a6aa29b4"} Sep 08 07:21:09.961349 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.961285 2571 scope.go:117] "RemoveContainer" containerID="23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a" Sep 08 07:21:09.970064 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.970047 2571 scope.go:117] "RemoveContainer" containerID="23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a" Sep 08 07:21:09.970366 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:21:09.970341 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a\": container with ID starting with 23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a not found: ID does not exist" containerID="23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a" Sep 08 07:21:09.970446 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:09.970373 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a"} err="failed to get container status \"23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a\": rpc error: code = NotFound desc = could not find container \"23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a\": container with ID starting with 23f61c74c81af4adad6ad222a3ee7bba90c3a9e02ea44b235f4db5453ba10e8a not found: ID does not exist" Sep 08 07:21:10.035414 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:10.035366 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-468m6\" (UniqueName: \"kubernetes.io/projected/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-kube-api-access-468m6\") pod \"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec\" (UID: \"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec\") " Sep 08 07:21:10.035614 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:10.035466 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-plugins-config-volume\") pod \"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec\" (UID: \"a7a5a069-dcdc-496c-9d85-20e6fc7db1ec\") " Sep 08 07:21:10.036131 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:10.036102 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7a5a069-dcdc-496c-9d85-20e6fc7db1ec" (UID: "a7a5a069-dcdc-496c-9d85-20e6fc7db1ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:21:10.037445 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:10.037421 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:21:10.038407 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:10.038380 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-kube-api-access-468m6" (OuterVolumeSpecName: "kube-api-access-468m6") pod "a7a5a069-dcdc-496c-9d85-20e6fc7db1ec" (UID: "a7a5a069-dcdc-496c-9d85-20e6fc7db1ec"). InnerVolumeSpecName "kube-api-access-468m6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:21:10.138605 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:10.138567 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-468m6\" (UniqueName: \"kubernetes.io/projected/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec-kube-api-access-468m6\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:21:10.284168 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:10.284134 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd"] Sep 08 07:21:10.290205 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:10.290177 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da54544b-749f95974b-xmtkd"] Sep 08 07:21:11.809246 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:11.809183 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7a5a069-dcdc-496c-9d85-20e6fc7db1ec" path="/var/lib/kubelet/pods/a7a5a069-dcdc-496c-9d85-20e6fc7db1ec/volumes" Sep 08 07:21:31.476622 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.476586 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w"] Sep 08 07:21:31.477004 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.476991 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92e88561-63a4-4ad3-85e7-dd214f290a8a" containerName="revoke-keys" Sep 08 07:21:31.477004 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.477005 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="92e88561-63a4-4ad3-85e7-dd214f290a8a" containerName="revoke-keys" Sep 08 07:21:31.477077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.477016 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" containerName="main" Sep 08 07:21:31.477077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.477022 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" containerName="main" Sep 08 07:21:31.477077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.477030 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a5a069-dcdc-496c-9d85-20e6fc7db1ec" containerName="payload-pre-processing" Sep 08 07:21:31.477077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.477036 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a5a069-dcdc-496c-9d85-20e6fc7db1ec" containerName="payload-pre-processing" Sep 08 07:21:31.477206 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.477089 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="91594dc2-1fa5-4fc3-afe2-e5dd4b8b66e8" containerName="main" Sep 08 07:21:31.477206 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.477095 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7a5a069-dcdc-496c-9d85-20e6fc7db1ec" containerName="payload-pre-processing" Sep 08 07:21:31.477206 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.477103 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="92e88561-63a4-4ad3-85e7-dd214f290a8a" containerName="revoke-keys" Sep 08 07:21:31.480149 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.480135 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:21:31.482569 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.482549 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-48a836\"" Sep 08 07:21:31.482684 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.482567 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-48a836-dockercfg-h7d25\"" Sep 08 07:21:31.490499 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.490474 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w"] Sep 08 07:21:31.543432 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.543400 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7hl5\" (UniqueName: \"kubernetes.io/projected/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-kube-api-access-l7hl5\") pod \"payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w\" (UID: \"7c83eaf1-cb79-4236-95c2-d8649b2f57c2\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:21:31.543595 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.543460 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w\" (UID: \"7c83eaf1-cb79-4236-95c2-d8649b2f57c2\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:21:31.644691 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.644654 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7hl5\" (UniqueName: \"kubernetes.io/projected/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-kube-api-access-l7hl5\") pod \"payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w\" (UID: \"7c83eaf1-cb79-4236-95c2-d8649b2f57c2\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:21:31.644861 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.644742 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w\" (UID: \"7c83eaf1-cb79-4236-95c2-d8649b2f57c2\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:21:31.645479 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.645459 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w\" (UID: \"7c83eaf1-cb79-4236-95c2-d8649b2f57c2\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:21:31.652631 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.652607 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7hl5\" (UniqueName: \"kubernetes.io/projected/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-kube-api-access-l7hl5\") pod \"payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w\" (UID: \"7c83eaf1-cb79-4236-95c2-d8649b2f57c2\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:21:31.791266 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:31.791154 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:21:32.062850 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:32.062813 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w"] Sep 08 07:21:32.063469 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:21:32.063442 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c83eaf1_cb79_4236_95c2_d8649b2f57c2.slice/crio-553bedda61b38c7a2dd8bca2f2edadf47d3ba548b15d4b971e5833d7a1458a40 WatchSource:0}: Error finding container 553bedda61b38c7a2dd8bca2f2edadf47d3ba548b15d4b971e5833d7a1458a40: Status 404 returned error can't find the container with id 553bedda61b38c7a2dd8bca2f2edadf47d3ba548b15d4b971e5833d7a1458a40 Sep 08 07:21:32.086072 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:32.086041 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" event={"ID":"7c83eaf1-cb79-4236-95c2-d8649b2f57c2","Type":"ContainerStarted","Data":"553bedda61b38c7a2dd8bca2f2edadf47d3ba548b15d4b971e5833d7a1458a40"} Sep 08 07:21:33.096774 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:33.096735 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" event={"ID":"7c83eaf1-cb79-4236-95c2-d8649b2f57c2","Type":"ContainerStarted","Data":"631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e"} Sep 08 07:21:33.097183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:33.096845 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:21:33.117223 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:33.117154 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" podStartSLOduration=2.117134862 podStartE2EDuration="2.117134862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:21:33.11369015 +0000 UTC m=+1007.791653289" watchObservedRunningTime="2026-09-08 07:21:33.117134862 +0000 UTC m=+1007.795097981" Sep 08 07:21:44.107373 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:44.107338 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:21:59.238660 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.238626 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm"] Sep 08 07:21:59.243313 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.243290 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.247006 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.246979 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-48a836\"/\"openshift-service-ca.crt\"" Sep 08 07:21:59.247138 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.247010 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-48a836\"/\"kube-root-ca.crt\"" Sep 08 07:21:59.247138 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.246979 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-48a836\"/\"default-dockercfg-bcwxg\"" Sep 08 07:21:59.247138 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.246979 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-48a836\"/\"explicit-model-48a836-kserve-self-signed-certs\"" Sep 08 07:21:59.252782 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.252761 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm"] Sep 08 07:21:59.407985 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.407944 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-dshm\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.407985 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.407990 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmn57\" (UniqueName: \"kubernetes.io/projected/a0770dac-a1ef-499a-8648-bd2719f8b134-kube-api-access-qmn57\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.408205 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.408105 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-tmp-dir\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.408205 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.408143 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-model-cache\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.408205 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.408186 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-home\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.408347 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.408272 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0770dac-a1ef-499a-8648-bd2719f8b134-tls-certs\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.509486 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.509385 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-model-cache\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.509486 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.509439 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-home\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.509715 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.509510 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0770dac-a1ef-499a-8648-bd2719f8b134-tls-certs\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.509715 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.509549 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-dshm\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.509715 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.509577 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmn57\" (UniqueName: \"kubernetes.io/projected/a0770dac-a1ef-499a-8648-bd2719f8b134-kube-api-access-qmn57\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.509715 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.509670 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-tmp-dir\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.509926 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.509842 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-model-cache\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.509926 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.509857 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-home\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.510072 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.510050 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-tmp-dir\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.512087 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.512062 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-dshm\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.512350 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.512332 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0770dac-a1ef-499a-8648-bd2719f8b134-tls-certs\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.517599 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.517576 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmn57\" (UniqueName: \"kubernetes.io/projected/a0770dac-a1ef-499a-8648-bd2719f8b134-kube-api-access-qmn57\") pod \"explicit-model-48a836-kserve-75747dd877-68rnm\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.554963 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.554929 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:21:59.826442 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:21:59.826412 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm"] Sep 08 07:21:59.827569 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:21:59.827546 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0770dac_a1ef_499a_8648_bd2719f8b134.slice/crio-b5f7f92285fbccb5866416d395b0849a654fa451713fcb55fb436657f11a53c4 WatchSource:0}: Error finding container b5f7f92285fbccb5866416d395b0849a654fa451713fcb55fb436657f11a53c4: Status 404 returned error can't find the container with id b5f7f92285fbccb5866416d395b0849a654fa451713fcb55fb436657f11a53c4 Sep 08 07:22:00.250834 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:00.250788 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" event={"ID":"a0770dac-a1ef-499a-8648-bd2719f8b134","Type":"ContainerStarted","Data":"1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823"} Sep 08 07:22:00.250834 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:00.250841 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" event={"ID":"a0770dac-a1ef-499a-8648-bd2719f8b134","Type":"ContainerStarted","Data":"b5f7f92285fbccb5866416d395b0849a654fa451713fcb55fb436657f11a53c4"} Sep 08 07:22:00.269565 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:00.269515 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" podStartSLOduration=1.269499454 podStartE2EDuration="1.269499454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:21:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:22:00.266509184 +0000 UTC m=+1034.944472321" watchObservedRunningTime="2026-09-08 07:22:00.269499454 +0000 UTC m=+1034.947462569" Sep 08 07:22:09.555167 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:09.555128 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:22:09.555653 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:09.555186 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:22:09.568007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:09.567980 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:22:10.320155 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:10.320126 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:22:17.382103 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:17.382067 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm"] Sep 08 07:22:17.497027 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:17.496995 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb"] Sep 08 07:22:17.500534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:17.500516 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb" Sep 08 07:22:17.503022 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:17.502998 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-824nh\"" Sep 08 07:22:17.503145 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:17.503000 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 07:22:17.507162 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:17.507142 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb"] Sep 08 07:22:17.585702 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:17.585667 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81ec4646-402c-4b47-9439-c2c94fccdea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-48a836-4gqnb\" (UID: \"81ec4646-402c-4b47-9439-c2c94fccdea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb" Sep 08 07:22:17.686728 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:17.686629 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81ec4646-402c-4b47-9439-c2c94fccdea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-48a836-4gqnb\" (UID: \"81ec4646-402c-4b47-9439-c2c94fccdea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb" Sep 08 07:22:17.687272 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:17.687254 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81ec4646-402c-4b47-9439-c2c94fccdea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-48a836-4gqnb\" (UID: \"81ec4646-402c-4b47-9439-c2c94fccdea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb" Sep 08 07:22:17.813317 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:17.813288 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb" Sep 08 07:22:18.084132 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:18.084106 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb"] Sep 08 07:22:18.085086 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:22:18.085059 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ec4646_402c_4b47_9439_c2c94fccdea7.slice/crio-c92c5d2cab8ecb6b2f34bf5eb9f6219b7a12f45b62ef96ec766a21d4d7d6203b WatchSource:0}: Error finding container c92c5d2cab8ecb6b2f34bf5eb9f6219b7a12f45b62ef96ec766a21d4d7d6203b: Status 404 returned error can't find the container with id c92c5d2cab8ecb6b2f34bf5eb9f6219b7a12f45b62ef96ec766a21d4d7d6203b Sep 08 07:22:18.355257 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:18.355160 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb" event={"ID":"81ec4646-402c-4b47-9439-c2c94fccdea7","Type":"ContainerStarted","Data":"c92c5d2cab8ecb6b2f34bf5eb9f6219b7a12f45b62ef96ec766a21d4d7d6203b"} Sep 08 07:22:19.362409 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:19.362365 2571 generic.go:356] "Generic (PLEG): container finished" podID="81ec4646-402c-4b47-9439-c2c94fccdea7" containerID="ce63f355b455ba5e72be52a35e9354a92f9904172595ff4480b77a475cd8e538" exitCode=0 Sep 08 07:22:19.362803 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:19.362427 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb" event={"ID":"81ec4646-402c-4b47-9439-c2c94fccdea7","Type":"ContainerDied","Data":"ce63f355b455ba5e72be52a35e9354a92f9904172595ff4480b77a475cd8e538"} Sep 08 07:22:20.497381 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:20.497355 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb" Sep 08 07:22:20.616352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:20.616321 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81ec4646-402c-4b47-9439-c2c94fccdea7-maas-api-service-ca\") pod \"81ec4646-402c-4b47-9439-c2c94fccdea7\" (UID: \"81ec4646-402c-4b47-9439-c2c94fccdea7\") " Sep 08 07:22:20.616640 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:20.616616 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81ec4646-402c-4b47-9439-c2c94fccdea7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "81ec4646-402c-4b47-9439-c2c94fccdea7" (UID: "81ec4646-402c-4b47-9439-c2c94fccdea7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:22:20.717474 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:20.717443 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81ec4646-402c-4b47-9439-c2c94fccdea7-maas-api-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:22:21.375141 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:21.375113 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb" Sep 08 07:22:21.375346 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:21.375145 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb" event={"ID":"81ec4646-402c-4b47-9439-c2c94fccdea7","Type":"ContainerDied","Data":"c92c5d2cab8ecb6b2f34bf5eb9f6219b7a12f45b62ef96ec766a21d4d7d6203b"} Sep 08 07:22:21.375346 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:21.375186 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c92c5d2cab8ecb6b2f34bf5eb9f6219b7a12f45b62ef96ec766a21d4d7d6203b" Sep 08 07:22:28.024372 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.024300 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w"] Sep 08 07:22:28.024843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.024618 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" podUID="7c83eaf1-cb79-4236-95c2-d8649b2f57c2" containerName="payload-pre-processing" containerID="cri-o://631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e" gracePeriod=30 Sep 08 07:22:28.188724 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.188686 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb"] Sep 08 07:22:28.191539 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.191507 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-48a836-4gqnb"] Sep 08 07:22:28.314943 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.314918 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:22:28.395836 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.395804 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7hl5\" (UniqueName: \"kubernetes.io/projected/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-kube-api-access-l7hl5\") pod \"7c83eaf1-cb79-4236-95c2-d8649b2f57c2\" (UID: \"7c83eaf1-cb79-4236-95c2-d8649b2f57c2\") " Sep 08 07:22:28.395836 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.395838 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-plugins-config-volume\") pod \"7c83eaf1-cb79-4236-95c2-d8649b2f57c2\" (UID: \"7c83eaf1-cb79-4236-95c2-d8649b2f57c2\") " Sep 08 07:22:28.396308 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.396283 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c83eaf1-cb79-4236-95c2-d8649b2f57c2" (UID: "7c83eaf1-cb79-4236-95c2-d8649b2f57c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:22:28.397926 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.397897 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-kube-api-access-l7hl5" (OuterVolumeSpecName: "kube-api-access-l7hl5") pod "7c83eaf1-cb79-4236-95c2-d8649b2f57c2" (UID: "7c83eaf1-cb79-4236-95c2-d8649b2f57c2"). InnerVolumeSpecName "kube-api-access-l7hl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:22:28.418670 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.418643 2571 generic.go:356] "Generic (PLEG): container finished" podID="7c83eaf1-cb79-4236-95c2-d8649b2f57c2" containerID="631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e" exitCode=0 Sep 08 07:22:28.418765 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.418689 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" Sep 08 07:22:28.418765 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.418723 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" event={"ID":"7c83eaf1-cb79-4236-95c2-d8649b2f57c2","Type":"ContainerDied","Data":"631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e"} Sep 08 07:22:28.418834 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.418777 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w" event={"ID":"7c83eaf1-cb79-4236-95c2-d8649b2f57c2","Type":"ContainerDied","Data":"553bedda61b38c7a2dd8bca2f2edadf47d3ba548b15d4b971e5833d7a1458a40"} Sep 08 07:22:28.418834 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.418808 2571 scope.go:117] "RemoveContainer" containerID="631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e" Sep 08 07:22:28.428002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.427982 2571 scope.go:117] "RemoveContainer" containerID="631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e" Sep 08 07:22:28.428297 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:22:28.428279 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e\": container with ID starting with 631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e not found: ID does not exist" containerID="631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e" Sep 08 07:22:28.428377 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.428306 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e"} err="failed to get container status \"631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e\": rpc error: code = NotFound desc = could not find container \"631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e\": container with ID starting with 631a8c9494403964ca5722cba76dfb6c86b943a3e4aefb9f957606a9764b268e not found: ID does not exist" Sep 08 07:22:28.441809 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.441789 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w"] Sep 08 07:22:28.446362 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.446342 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-48a836-869cf66dd6-9lv9w"] Sep 08 07:22:28.497617 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.497590 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7hl5\" (UniqueName: \"kubernetes.io/projected/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-kube-api-access-l7hl5\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:22:28.497617 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:28.497612 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c83eaf1-cb79-4236-95c2-d8649b2f57c2-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:22:29.809780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:29.809748 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c83eaf1-cb79-4236-95c2-d8649b2f57c2" path="/var/lib/kubelet/pods/7c83eaf1-cb79-4236-95c2-d8649b2f57c2/volumes" Sep 08 07:22:29.810138 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:29.810059 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81ec4646-402c-4b47-9439-c2c94fccdea7" path="/var/lib/kubelet/pods/81ec4646-402c-4b47-9439-c2c94fccdea7/volumes" Sep 08 07:22:32.402267 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.402181 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" podUID="a0770dac-a1ef-499a-8648-bd2719f8b134" containerName="main" containerID="cri-o://1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823" gracePeriod=45 Sep 08 07:22:32.651172 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.651144 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:22:32.743676 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.743629 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-model-cache\") pod \"a0770dac-a1ef-499a-8648-bd2719f8b134\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " Sep 08 07:22:32.743881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.743726 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-tmp-dir\") pod \"a0770dac-a1ef-499a-8648-bd2719f8b134\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " Sep 08 07:22:32.743881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.743770 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-dshm\") pod \"a0770dac-a1ef-499a-8648-bd2719f8b134\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " Sep 08 07:22:32.743881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.743793 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-home\") pod \"a0770dac-a1ef-499a-8648-bd2719f8b134\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " Sep 08 07:22:32.743881 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.743837 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmn57\" (UniqueName: \"kubernetes.io/projected/a0770dac-a1ef-499a-8648-bd2719f8b134-kube-api-access-qmn57\") pod \"a0770dac-a1ef-499a-8648-bd2719f8b134\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " Sep 08 07:22:32.744077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.743887 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0770dac-a1ef-499a-8648-bd2719f8b134-tls-certs\") pod \"a0770dac-a1ef-499a-8648-bd2719f8b134\" (UID: \"a0770dac-a1ef-499a-8648-bd2719f8b134\") " Sep 08 07:22:32.744077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.743934 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-model-cache" (OuterVolumeSpecName: "model-cache") pod "a0770dac-a1ef-499a-8648-bd2719f8b134" (UID: "a0770dac-a1ef-499a-8648-bd2719f8b134"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:22:32.744077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.744011 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a0770dac-a1ef-499a-8648-bd2719f8b134" (UID: "a0770dac-a1ef-499a-8648-bd2719f8b134"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:22:32.744077 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.744051 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-home" (OuterVolumeSpecName: "home") pod "a0770dac-a1ef-499a-8648-bd2719f8b134" (UID: "a0770dac-a1ef-499a-8648-bd2719f8b134"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:22:32.744309 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.744291 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-tmp-dir\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:22:32.744309 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.744306 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-home\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:22:32.744429 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.744325 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-model-cache\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:22:32.746000 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.745971 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-dshm" (OuterVolumeSpecName: "dshm") pod "a0770dac-a1ef-499a-8648-bd2719f8b134" (UID: "a0770dac-a1ef-499a-8648-bd2719f8b134"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:22:32.746096 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.746030 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0770dac-a1ef-499a-8648-bd2719f8b134-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a0770dac-a1ef-499a-8648-bd2719f8b134" (UID: "a0770dac-a1ef-499a-8648-bd2719f8b134"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:22:32.746096 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.746068 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0770dac-a1ef-499a-8648-bd2719f8b134-kube-api-access-qmn57" (OuterVolumeSpecName: "kube-api-access-qmn57") pod "a0770dac-a1ef-499a-8648-bd2719f8b134" (UID: "a0770dac-a1ef-499a-8648-bd2719f8b134"). InnerVolumeSpecName "kube-api-access-qmn57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:22:32.845224 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.845190 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0770dac-a1ef-499a-8648-bd2719f8b134-dshm\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:22:32.845388 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.845238 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmn57\" (UniqueName: \"kubernetes.io/projected/a0770dac-a1ef-499a-8648-bd2719f8b134-kube-api-access-qmn57\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:22:32.845388 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:32.845254 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0770dac-a1ef-499a-8648-bd2719f8b134-tls-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:22:33.450023 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:33.449991 2571 generic.go:356] "Generic (PLEG): container finished" podID="a0770dac-a1ef-499a-8648-bd2719f8b134" containerID="1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823" exitCode=0 Sep 08 07:22:33.450464 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:33.450040 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" Sep 08 07:22:33.450464 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:33.450070 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" event={"ID":"a0770dac-a1ef-499a-8648-bd2719f8b134","Type":"ContainerDied","Data":"1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823"} Sep 08 07:22:33.450464 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:33.450112 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm" event={"ID":"a0770dac-a1ef-499a-8648-bd2719f8b134","Type":"ContainerDied","Data":"b5f7f92285fbccb5866416d395b0849a654fa451713fcb55fb436657f11a53c4"} Sep 08 07:22:33.450464 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:33.450129 2571 scope.go:117] "RemoveContainer" containerID="1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823" Sep 08 07:22:33.459592 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:33.459572 2571 scope.go:117] "RemoveContainer" containerID="1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823" Sep 08 07:22:33.459856 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:22:33.459837 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823\": container with ID starting with 1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823 not found: ID does not exist" containerID="1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823" Sep 08 07:22:33.459945 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:33.459866 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823"} err="failed to get container status \"1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823\": rpc error: code = NotFound desc = could not find container \"1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823\": container with ID starting with 1e1ebaa2d27d81768dad6e83393b2bcf76b62894bad7f5f4e38acd91d14a5823 not found: ID does not exist" Sep 08 07:22:33.471761 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:33.471735 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm"] Sep 08 07:22:33.475335 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:33.475314 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-48a836/explicit-model-48a836-kserve-75747dd877-68rnm"] Sep 08 07:22:33.809455 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:33.809376 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0770dac-a1ef-499a-8648-bd2719f8b134" path="/var/lib/kubelet/pods/a0770dac-a1ef-499a-8648-bd2719f8b134/volumes" Sep 08 07:22:49.951416 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.951377 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd"] Sep 08 07:22:49.951874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.951763 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81ec4646-402c-4b47-9439-c2c94fccdea7" containerName="revoke-keys" Sep 08 07:22:49.951874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.951775 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="81ec4646-402c-4b47-9439-c2c94fccdea7" containerName="revoke-keys" Sep 08 07:22:49.951874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.951786 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0770dac-a1ef-499a-8648-bd2719f8b134" containerName="main" Sep 08 07:22:49.951874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.951792 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0770dac-a1ef-499a-8648-bd2719f8b134" containerName="main" Sep 08 07:22:49.951874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.951818 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c83eaf1-cb79-4236-95c2-d8649b2f57c2" containerName="payload-pre-processing" Sep 08 07:22:49.951874 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.951824 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c83eaf1-cb79-4236-95c2-d8649b2f57c2" containerName="payload-pre-processing" Sep 08 07:22:49.952063 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.951882 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0770dac-a1ef-499a-8648-bd2719f8b134" containerName="main" Sep 08 07:22:49.952063 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.951893 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c83eaf1-cb79-4236-95c2-d8649b2f57c2" containerName="payload-pre-processing" Sep 08 07:22:49.952063 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.951900 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="81ec4646-402c-4b47-9439-c2c94fccdea7" containerName="revoke-keys" Sep 08 07:22:49.956547 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.956526 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:22:49.958986 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.958962 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9988a479\"" Sep 08 07:22:49.966563 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.966535 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd"] Sep 08 07:22:49.984515 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.984472 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk"] Sep 08 07:22:49.988720 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.988697 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:22:49.991138 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.991102 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9988a479-dockercfg-2dpb4\"" Sep 08 07:22:49.995308 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:49.995283 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk"] Sep 08 07:22:50.003639 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.003614 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b231ab5-8599-4ec2-b5fc-7818000cc341-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd\" (UID: \"6b231ab5-8599-4ec2-b5fc-7818000cc341\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:22:50.003792 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.003665 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjfgk\" (UniqueName: \"kubernetes.io/projected/6c4ea1b3-6e99-4f71-943b-70158c5a69af-kube-api-access-xjfgk\") pod \"payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk\" (UID: \"6c4ea1b3-6e99-4f71-943b-70158c5a69af\") " pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:22:50.003866 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.003799 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4ea1b3-6e99-4f71-943b-70158c5a69af-plugins-config-volume\") pod \"payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk\" (UID: \"6c4ea1b3-6e99-4f71-943b-70158c5a69af\") " pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:22:50.003866 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.003837 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czlm4\" (UniqueName: \"kubernetes.io/projected/6b231ab5-8599-4ec2-b5fc-7818000cc341-kube-api-access-czlm4\") pod \"payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd\" (UID: \"6b231ab5-8599-4ec2-b5fc-7818000cc341\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:22:50.104624 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.104585 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4ea1b3-6e99-4f71-943b-70158c5a69af-plugins-config-volume\") pod \"payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk\" (UID: \"6c4ea1b3-6e99-4f71-943b-70158c5a69af\") " pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:22:50.104808 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.104635 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czlm4\" (UniqueName: \"kubernetes.io/projected/6b231ab5-8599-4ec2-b5fc-7818000cc341-kube-api-access-czlm4\") pod \"payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd\" (UID: \"6b231ab5-8599-4ec2-b5fc-7818000cc341\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:22:50.104808 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.104683 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b231ab5-8599-4ec2-b5fc-7818000cc341-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd\" (UID: \"6b231ab5-8599-4ec2-b5fc-7818000cc341\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:22:50.104808 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.104727 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjfgk\" (UniqueName: \"kubernetes.io/projected/6c4ea1b3-6e99-4f71-943b-70158c5a69af-kube-api-access-xjfgk\") pod \"payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk\" (UID: \"6c4ea1b3-6e99-4f71-943b-70158c5a69af\") " pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:22:50.105570 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.105546 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b231ab5-8599-4ec2-b5fc-7818000cc341-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd\" (UID: \"6b231ab5-8599-4ec2-b5fc-7818000cc341\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:22:50.105648 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.105546 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4ea1b3-6e99-4f71-943b-70158c5a69af-plugins-config-volume\") pod \"payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk\" (UID: \"6c4ea1b3-6e99-4f71-943b-70158c5a69af\") " pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:22:50.117039 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.117003 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjfgk\" (UniqueName: \"kubernetes.io/projected/6c4ea1b3-6e99-4f71-943b-70158c5a69af-kube-api-access-xjfgk\") pod \"payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk\" (UID: \"6c4ea1b3-6e99-4f71-943b-70158c5a69af\") " pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:22:50.117164 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.117084 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czlm4\" (UniqueName: \"kubernetes.io/projected/6b231ab5-8599-4ec2-b5fc-7818000cc341-kube-api-access-czlm4\") pod \"payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd\" (UID: \"6b231ab5-8599-4ec2-b5fc-7818000cc341\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:22:50.267961 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.267863 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:22:50.301390 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.301358 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:22:50.543683 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.543655 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd"] Sep 08 07:22:50.544562 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:22:50.544534 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b231ab5_8599_4ec2_b5fc_7818000cc341.slice/crio-4a75bacad8b933eac2e88f24a6d88c129c828c9a145ff84bb65910bd8672a844 WatchSource:0}: Error finding container 4a75bacad8b933eac2e88f24a6d88c129c828c9a145ff84bb65910bd8672a844: Status 404 returned error can't find the container with id 4a75bacad8b933eac2e88f24a6d88c129c828c9a145ff84bb65910bd8672a844 Sep 08 07:22:50.585933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:50.585899 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk"] Sep 08 07:22:50.603406 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:22:50.603367 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4ea1b3_6e99_4f71_943b_70158c5a69af.slice/crio-11fe968a6d092a6d38a438013d22e14f132ca61268d4b6cb158487087ab3e8d8 WatchSource:0}: Error finding container 11fe968a6d092a6d38a438013d22e14f132ca61268d4b6cb158487087ab3e8d8: Status 404 returned error can't find the container with id 11fe968a6d092a6d38a438013d22e14f132ca61268d4b6cb158487087ab3e8d8 Sep 08 07:22:51.557762 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:51.557729 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" event={"ID":"6b231ab5-8599-4ec2-b5fc-7818000cc341","Type":"ContainerStarted","Data":"910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de"} Sep 08 07:22:51.558241 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:51.557782 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" event={"ID":"6b231ab5-8599-4ec2-b5fc-7818000cc341","Type":"ContainerStarted","Data":"4a75bacad8b933eac2e88f24a6d88c129c828c9a145ff84bb65910bd8672a844"} Sep 08 07:22:51.558241 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:51.557828 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:22:51.559327 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:51.559305 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" event={"ID":"6c4ea1b3-6e99-4f71-943b-70158c5a69af","Type":"ContainerStarted","Data":"80b177fb451a3c0120f97223075c37ddfb524e2c02b39ecdcced203090c73e28"} Sep 08 07:22:51.559424 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:51.559347 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" event={"ID":"6c4ea1b3-6e99-4f71-943b-70158c5a69af","Type":"ContainerStarted","Data":"11fe968a6d092a6d38a438013d22e14f132ca61268d4b6cb158487087ab3e8d8"} Sep 08 07:22:51.559485 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:51.559471 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:22:51.574102 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:51.574052 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" podStartSLOduration=2.574033255 podStartE2EDuration="2.574033255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:22:51.573150466 +0000 UTC m=+1086.251113608" watchObservedRunningTime="2026-09-08 07:22:51.574033255 +0000 UTC m=+1086.251996374" Sep 08 07:22:51.590981 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:22:51.590926 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" podStartSLOduration=2.5909123689999998 podStartE2EDuration="2.590912369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:22:51.588182661 +0000 UTC m=+1086.266145799" watchObservedRunningTime="2026-09-08 07:22:51.590912369 +0000 UTC m=+1086.268875484" Sep 08 07:23:02.567121 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:02.567088 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:23:02.567517 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:02.567256 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:23:20.915830 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:20.915791 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75"] Sep 08 07:23:20.919036 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:20.919020 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" Sep 08 07:23:20.921406 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:20.921384 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 07:23:20.921505 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:20.921388 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-824nh\"" Sep 08 07:23:20.927247 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:20.927156 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75"] Sep 08 07:23:21.011826 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:21.011788 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30ed99a9-84bd-450c-a6e7-ee56626c664a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9988a479-f6p75\" (UID: \"30ed99a9-84bd-450c-a6e7-ee56626c664a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" Sep 08 07:23:21.113275 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:21.113238 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30ed99a9-84bd-450c-a6e7-ee56626c664a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9988a479-f6p75\" (UID: \"30ed99a9-84bd-450c-a6e7-ee56626c664a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" Sep 08 07:23:21.113875 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:21.113854 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30ed99a9-84bd-450c-a6e7-ee56626c664a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9988a479-f6p75\" (UID: \"30ed99a9-84bd-450c-a6e7-ee56626c664a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" Sep 08 07:23:21.230414 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:21.230367 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" Sep 08 07:23:21.502939 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:21.502908 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75"] Sep 08 07:23:21.504244 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:23:21.504198 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ed99a9_84bd_450c_a6e7_ee56626c664a.slice/crio-586d1f495d2ccfce2ebaa7b2c4dd24afca5a73509cb29315a0b4a4660569766c WatchSource:0}: Error finding container 586d1f495d2ccfce2ebaa7b2c4dd24afca5a73509cb29315a0b4a4660569766c: Status 404 returned error can't find the container with id 586d1f495d2ccfce2ebaa7b2c4dd24afca5a73509cb29315a0b4a4660569766c Sep 08 07:23:21.740475 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:21.740437 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" event={"ID":"30ed99a9-84bd-450c-a6e7-ee56626c664a","Type":"ContainerStarted","Data":"cca46e5b0a2d08c1e98160a5ce874a420959120981a6a73e6bb41df304a97007"} Sep 08 07:23:21.740638 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:21.740499 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" event={"ID":"30ed99a9-84bd-450c-a6e7-ee56626c664a","Type":"ContainerStarted","Data":"586d1f495d2ccfce2ebaa7b2c4dd24afca5a73509cb29315a0b4a4660569766c"} Sep 08 07:23:21.758799 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:21.758716 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" podStartSLOduration=1.7587004899999998 podStartE2EDuration="1.75870049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:23:21.755618175 +0000 UTC m=+1116.433581309" watchObservedRunningTime="2026-09-08 07:23:21.75870049 +0000 UTC m=+1116.436663606" Sep 08 07:23:22.751518 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:22.751484 2571 generic.go:356] "Generic (PLEG): container finished" podID="30ed99a9-84bd-450c-a6e7-ee56626c664a" containerID="cca46e5b0a2d08c1e98160a5ce874a420959120981a6a73e6bb41df304a97007" exitCode=0 Sep 08 07:23:22.751893 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:22.751557 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" event={"ID":"30ed99a9-84bd-450c-a6e7-ee56626c664a","Type":"ContainerDied","Data":"cca46e5b0a2d08c1e98160a5ce874a420959120981a6a73e6bb41df304a97007"} Sep 08 07:23:23.895415 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:23.895387 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" Sep 08 07:23:23.942573 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:23.942526 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30ed99a9-84bd-450c-a6e7-ee56626c664a-maas-api-service-ca\") pod \"30ed99a9-84bd-450c-a6e7-ee56626c664a\" (UID: \"30ed99a9-84bd-450c-a6e7-ee56626c664a\") " Sep 08 07:23:23.942996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:23.942965 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30ed99a9-84bd-450c-a6e7-ee56626c664a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "30ed99a9-84bd-450c-a6e7-ee56626c664a" (UID: "30ed99a9-84bd-450c-a6e7-ee56626c664a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:23:24.043813 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:24.043728 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30ed99a9-84bd-450c-a6e7-ee56626c664a-maas-api-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:24.764803 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:24.764775 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" Sep 08 07:23:24.764984 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:24.764811 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75" event={"ID":"30ed99a9-84bd-450c-a6e7-ee56626c664a","Type":"ContainerDied","Data":"586d1f495d2ccfce2ebaa7b2c4dd24afca5a73509cb29315a0b4a4660569766c"} Sep 08 07:23:24.764984 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:24.764883 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="586d1f495d2ccfce2ebaa7b2c4dd24afca5a73509cb29315a0b4a4660569766c" Sep 08 07:23:30.057795 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.057759 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd"] Sep 08 07:23:30.058186 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.058033 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" podUID="943a64b8-2f78-459a-b319-5cda46aaea80" containerName="payload-processing" containerID="cri-o://a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492" gracePeriod=30 Sep 08 07:23:30.067973 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.067931 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z"] Sep 08 07:23:30.068401 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.068357 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" podUID="981ec975-bd6c-4e89-9b03-b3e2339fe0da" containerName="payload-pre-processing" containerID="cri-o://18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9" gracePeriod=30 Sep 08 07:23:30.357789 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.357765 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:23:30.360550 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.360529 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:23:30.408437 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.408398 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/943a64b8-2f78-459a-b319-5cda46aaea80-plugins-config-volume\") pod \"943a64b8-2f78-459a-b319-5cda46aaea80\" (UID: \"943a64b8-2f78-459a-b319-5cda46aaea80\") " Sep 08 07:23:30.408597 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.408536 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj4v6\" (UniqueName: \"kubernetes.io/projected/943a64b8-2f78-459a-b319-5cda46aaea80-kube-api-access-sj4v6\") pod \"943a64b8-2f78-459a-b319-5cda46aaea80\" (UID: \"943a64b8-2f78-459a-b319-5cda46aaea80\") " Sep 08 07:23:30.408651 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.408619 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/981ec975-bd6c-4e89-9b03-b3e2339fe0da-plugins-config-volume\") pod \"981ec975-bd6c-4e89-9b03-b3e2339fe0da\" (UID: \"981ec975-bd6c-4e89-9b03-b3e2339fe0da\") " Sep 08 07:23:30.408699 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.408649 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9s6t7\" (UniqueName: \"kubernetes.io/projected/981ec975-bd6c-4e89-9b03-b3e2339fe0da-kube-api-access-9s6t7\") pod \"981ec975-bd6c-4e89-9b03-b3e2339fe0da\" (UID: \"981ec975-bd6c-4e89-9b03-b3e2339fe0da\") " Sep 08 07:23:30.408771 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.408744 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/943a64b8-2f78-459a-b319-5cda46aaea80-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "943a64b8-2f78-459a-b319-5cda46aaea80" (UID: "943a64b8-2f78-459a-b319-5cda46aaea80"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:23:30.409028 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.409004 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/943a64b8-2f78-459a-b319-5cda46aaea80-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:30.409158 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.409037 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/981ec975-bd6c-4e89-9b03-b3e2339fe0da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "981ec975-bd6c-4e89-9b03-b3e2339fe0da" (UID: "981ec975-bd6c-4e89-9b03-b3e2339fe0da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:23:30.410761 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.410736 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/943a64b8-2f78-459a-b319-5cda46aaea80-kube-api-access-sj4v6" (OuterVolumeSpecName: "kube-api-access-sj4v6") pod "943a64b8-2f78-459a-b319-5cda46aaea80" (UID: "943a64b8-2f78-459a-b319-5cda46aaea80"). InnerVolumeSpecName "kube-api-access-sj4v6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:23:30.410865 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.410771 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/981ec975-bd6c-4e89-9b03-b3e2339fe0da-kube-api-access-9s6t7" (OuterVolumeSpecName: "kube-api-access-9s6t7") pod "981ec975-bd6c-4e89-9b03-b3e2339fe0da" (UID: "981ec975-bd6c-4e89-9b03-b3e2339fe0da"). InnerVolumeSpecName "kube-api-access-9s6t7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:23:30.509525 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.509491 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/981ec975-bd6c-4e89-9b03-b3e2339fe0da-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:30.509525 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.509519 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s6t7\" (UniqueName: \"kubernetes.io/projected/981ec975-bd6c-4e89-9b03-b3e2339fe0da-kube-api-access-9s6t7\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:30.509525 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.509529 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj4v6\" (UniqueName: \"kubernetes.io/projected/943a64b8-2f78-459a-b319-5cda46aaea80-kube-api-access-sj4v6\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:30.804106 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.804065 2571 generic.go:356] "Generic (PLEG): container finished" podID="981ec975-bd6c-4e89-9b03-b3e2339fe0da" containerID="18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9" exitCode=0 Sep 08 07:23:30.804331 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.804128 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" Sep 08 07:23:30.804331 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.804149 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" event={"ID":"981ec975-bd6c-4e89-9b03-b3e2339fe0da","Type":"ContainerDied","Data":"18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9"} Sep 08 07:23:30.804331 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.804200 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z" event={"ID":"981ec975-bd6c-4e89-9b03-b3e2339fe0da","Type":"ContainerDied","Data":"49f6520587ab619e4b2d8bdf198679231febf368b1b3cce68d858442a6f675dc"} Sep 08 07:23:30.804331 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.804237 2571 scope.go:117] "RemoveContainer" containerID="18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9" Sep 08 07:23:30.805523 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.805495 2571 generic.go:356] "Generic (PLEG): container finished" podID="943a64b8-2f78-459a-b319-5cda46aaea80" containerID="a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492" exitCode=0 Sep 08 07:23:30.805611 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.805535 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" event={"ID":"943a64b8-2f78-459a-b319-5cda46aaea80","Type":"ContainerDied","Data":"a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492"} Sep 08 07:23:30.805611 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.805566 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" Sep 08 07:23:30.805707 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.805569 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd" event={"ID":"943a64b8-2f78-459a-b319-5cda46aaea80","Type":"ContainerDied","Data":"7b02a31de450156eab3748a7c4f8373d5bc83d71e67be253bae0e1c2ed88cecb"} Sep 08 07:23:30.813503 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.813483 2571 scope.go:117] "RemoveContainer" containerID="18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9" Sep 08 07:23:30.813796 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:23:30.813771 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9\": container with ID starting with 18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9 not found: ID does not exist" containerID="18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9" Sep 08 07:23:30.813883 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.813803 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9"} err="failed to get container status \"18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9\": rpc error: code = NotFound desc = could not find container \"18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9\": container with ID starting with 18318929ff208c8757f1f178182a3d5a385dda65d8e63d86554a71a078b1b4e9 not found: ID does not exist" Sep 08 07:23:30.813883 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.813827 2571 scope.go:117] "RemoveContainer" containerID="a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492" Sep 08 07:23:30.822157 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.822141 2571 scope.go:117] "RemoveContainer" containerID="a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492" Sep 08 07:23:30.822412 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:23:30.822394 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492\": container with ID starting with a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492 not found: ID does not exist" containerID="a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492" Sep 08 07:23:30.822466 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.822419 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492"} err="failed to get container status \"a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492\": rpc error: code = NotFound desc = could not find container \"a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492\": container with ID starting with a2f6795edc7d9482b88e6064b177545497c0ed8951a2bf6d542d7eca2b77f492 not found: ID does not exist" Sep 08 07:23:30.830529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.830508 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z"] Sep 08 07:23:30.834046 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.834025 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-620112-8586456674-bbz9z"] Sep 08 07:23:30.845303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.845280 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd"] Sep 08 07:23:30.850807 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:30.850788 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-620112-66bfcfdbb9-xlsnd"] Sep 08 07:23:31.693864 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.693825 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk"] Sep 08 07:23:31.694354 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.694072 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" podUID="6c4ea1b3-6e99-4f71-943b-70158c5a69af" containerName="payload-processing" containerID="cri-o://80b177fb451a3c0120f97223075c37ddfb524e2c02b39ecdcced203090c73e28" gracePeriod=30 Sep 08 07:23:31.700323 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.700281 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd"] Sep 08 07:23:31.700701 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.700671 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" podUID="6b231ab5-8599-4ec2-b5fc-7818000cc341" containerName="payload-pre-processing" containerID="cri-o://910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de" gracePeriod=30 Sep 08 07:23:31.810598 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.810555 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="943a64b8-2f78-459a-b319-5cda46aaea80" path="/var/lib/kubelet/pods/943a64b8-2f78-459a-b319-5cda46aaea80/volumes" Sep 08 07:23:31.811066 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.811046 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="981ec975-bd6c-4e89-9b03-b3e2339fe0da" path="/var/lib/kubelet/pods/981ec975-bd6c-4e89-9b03-b3e2339fe0da/volumes" Sep 08 07:23:31.813443 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.813422 2571 generic.go:356] "Generic (PLEG): container finished" podID="6c4ea1b3-6e99-4f71-943b-70158c5a69af" containerID="80b177fb451a3c0120f97223075c37ddfb524e2c02b39ecdcced203090c73e28" exitCode=0 Sep 08 07:23:31.813556 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.813492 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" event={"ID":"6c4ea1b3-6e99-4f71-943b-70158c5a69af","Type":"ContainerDied","Data":"80b177fb451a3c0120f97223075c37ddfb524e2c02b39ecdcced203090c73e28"} Sep 08 07:23:31.867770 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.867672 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75"] Sep 08 07:23:31.870226 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.870192 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9988a479-f6p75"] Sep 08 07:23:31.967820 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.967797 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:23:31.970728 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:31.970710 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:23:32.025528 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.025139 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4ea1b3-6e99-4f71-943b-70158c5a69af-plugins-config-volume\") pod \"6c4ea1b3-6e99-4f71-943b-70158c5a69af\" (UID: \"6c4ea1b3-6e99-4f71-943b-70158c5a69af\") " Sep 08 07:23:32.025528 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.025282 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjfgk\" (UniqueName: \"kubernetes.io/projected/6c4ea1b3-6e99-4f71-943b-70158c5a69af-kube-api-access-xjfgk\") pod \"6c4ea1b3-6e99-4f71-943b-70158c5a69af\" (UID: \"6c4ea1b3-6e99-4f71-943b-70158c5a69af\") " Sep 08 07:23:32.025528 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.025317 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czlm4\" (UniqueName: \"kubernetes.io/projected/6b231ab5-8599-4ec2-b5fc-7818000cc341-kube-api-access-czlm4\") pod \"6b231ab5-8599-4ec2-b5fc-7818000cc341\" (UID: \"6b231ab5-8599-4ec2-b5fc-7818000cc341\") " Sep 08 07:23:32.025528 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.025367 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b231ab5-8599-4ec2-b5fc-7818000cc341-plugins-config-volume\") pod \"6b231ab5-8599-4ec2-b5fc-7818000cc341\" (UID: \"6b231ab5-8599-4ec2-b5fc-7818000cc341\") " Sep 08 07:23:32.026144 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.026099 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b231ab5-8599-4ec2-b5fc-7818000cc341-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b231ab5-8599-4ec2-b5fc-7818000cc341" (UID: "6b231ab5-8599-4ec2-b5fc-7818000cc341"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:23:32.026271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.026164 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c4ea1b3-6e99-4f71-943b-70158c5a69af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6c4ea1b3-6e99-4f71-943b-70158c5a69af" (UID: "6c4ea1b3-6e99-4f71-943b-70158c5a69af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:23:32.029372 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.029189 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c4ea1b3-6e99-4f71-943b-70158c5a69af-kube-api-access-xjfgk" (OuterVolumeSpecName: "kube-api-access-xjfgk") pod "6c4ea1b3-6e99-4f71-943b-70158c5a69af" (UID: "6c4ea1b3-6e99-4f71-943b-70158c5a69af"). InnerVolumeSpecName "kube-api-access-xjfgk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:23:32.029372 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.029326 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b231ab5-8599-4ec2-b5fc-7818000cc341-kube-api-access-czlm4" (OuterVolumeSpecName: "kube-api-access-czlm4") pod "6b231ab5-8599-4ec2-b5fc-7818000cc341" (UID: "6b231ab5-8599-4ec2-b5fc-7818000cc341"). InnerVolumeSpecName "kube-api-access-czlm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:23:32.126697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.126657 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4ea1b3-6e99-4f71-943b-70158c5a69af-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:32.126697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.126691 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjfgk\" (UniqueName: \"kubernetes.io/projected/6c4ea1b3-6e99-4f71-943b-70158c5a69af-kube-api-access-xjfgk\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:32.126901 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.126706 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czlm4\" (UniqueName: \"kubernetes.io/projected/6b231ab5-8599-4ec2-b5fc-7818000cc341-kube-api-access-czlm4\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:32.126901 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.126718 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b231ab5-8599-4ec2-b5fc-7818000cc341-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:32.822771 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.822728 2571 generic.go:356] "Generic (PLEG): container finished" podID="6b231ab5-8599-4ec2-b5fc-7818000cc341" containerID="910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de" exitCode=0 Sep 08 07:23:32.823263 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.822790 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" Sep 08 07:23:32.823263 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.822820 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" event={"ID":"6b231ab5-8599-4ec2-b5fc-7818000cc341","Type":"ContainerDied","Data":"910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de"} Sep 08 07:23:32.823263 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.822871 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd" event={"ID":"6b231ab5-8599-4ec2-b5fc-7818000cc341","Type":"ContainerDied","Data":"4a75bacad8b933eac2e88f24a6d88c129c828c9a145ff84bb65910bd8672a844"} Sep 08 07:23:32.823263 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.822889 2571 scope.go:117] "RemoveContainer" containerID="910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de" Sep 08 07:23:32.824441 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.824420 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" Sep 08 07:23:32.824559 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.824451 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk" event={"ID":"6c4ea1b3-6e99-4f71-943b-70158c5a69af","Type":"ContainerDied","Data":"11fe968a6d092a6d38a438013d22e14f132ca61268d4b6cb158487087ab3e8d8"} Sep 08 07:23:32.832811 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.832793 2571 scope.go:117] "RemoveContainer" containerID="910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de" Sep 08 07:23:32.833088 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:23:32.833065 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de\": container with ID starting with 910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de not found: ID does not exist" containerID="910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de" Sep 08 07:23:32.833190 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.833100 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de"} err="failed to get container status \"910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de\": rpc error: code = NotFound desc = could not find container \"910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de\": container with ID starting with 910e44e7af1a79ccd849e576ae5ca3432675e8156f85585978f47aef0543f2de not found: ID does not exist" Sep 08 07:23:32.833190 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.833123 2571 scope.go:117] "RemoveContainer" containerID="80b177fb451a3c0120f97223075c37ddfb524e2c02b39ecdcced203090c73e28" Sep 08 07:23:32.846128 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.846106 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd"] Sep 08 07:23:32.850694 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.850672 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9988a479-9ddc7c78b-ddpnd"] Sep 08 07:23:32.861841 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.861803 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk"] Sep 08 07:23:32.864814 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:32.864795 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9988a479-7bcd6d5cb5-bw6xk"] Sep 08 07:23:33.809997 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:33.809965 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30ed99a9-84bd-450c-a6e7-ee56626c664a" path="/var/lib/kubelet/pods/30ed99a9-84bd-450c-a6e7-ee56626c664a/volumes" Sep 08 07:23:33.810376 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:33.810362 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b231ab5-8599-4ec2-b5fc-7818000cc341" path="/var/lib/kubelet/pods/6b231ab5-8599-4ec2-b5fc-7818000cc341/volumes" Sep 08 07:23:33.810707 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:33.810695 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c4ea1b3-6e99-4f71-943b-70158c5a69af" path="/var/lib/kubelet/pods/6c4ea1b3-6e99-4f71-943b-70158c5a69af/volumes" Sep 08 07:23:37.324014 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.323971 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7"] Sep 08 07:23:37.324529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.324274 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" containerID="cri-o://1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0" gracePeriod=30 Sep 08 07:23:37.651380 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651304 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4"] Sep 08 07:23:37.651691 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651677 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="981ec975-bd6c-4e89-9b03-b3e2339fe0da" containerName="payload-pre-processing" Sep 08 07:23:37.651738 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651693 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="981ec975-bd6c-4e89-9b03-b3e2339fe0da" containerName="payload-pre-processing" Sep 08 07:23:37.651738 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651703 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c4ea1b3-6e99-4f71-943b-70158c5a69af" containerName="payload-processing" Sep 08 07:23:37.651738 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651709 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c4ea1b3-6e99-4f71-943b-70158c5a69af" containerName="payload-processing" Sep 08 07:23:37.651738 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651718 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="943a64b8-2f78-459a-b319-5cda46aaea80" containerName="payload-processing" Sep 08 07:23:37.651738 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651724 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="943a64b8-2f78-459a-b319-5cda46aaea80" containerName="payload-processing" Sep 08 07:23:37.651738 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651731 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b231ab5-8599-4ec2-b5fc-7818000cc341" containerName="payload-pre-processing" Sep 08 07:23:37.651738 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651737 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b231ab5-8599-4ec2-b5fc-7818000cc341" containerName="payload-pre-processing" Sep 08 07:23:37.651942 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651745 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30ed99a9-84bd-450c-a6e7-ee56626c664a" containerName="revoke-keys" Sep 08 07:23:37.651942 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651750 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="30ed99a9-84bd-450c-a6e7-ee56626c664a" containerName="revoke-keys" Sep 08 07:23:37.651942 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651810 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="30ed99a9-84bd-450c-a6e7-ee56626c664a" containerName="revoke-keys" Sep 08 07:23:37.651942 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651818 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b231ab5-8599-4ec2-b5fc-7818000cc341" containerName="payload-pre-processing" Sep 08 07:23:37.651942 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651827 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c4ea1b3-6e99-4f71-943b-70158c5a69af" containerName="payload-processing" Sep 08 07:23:37.651942 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651833 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="943a64b8-2f78-459a-b319-5cda46aaea80" containerName="payload-processing" Sep 08 07:23:37.651942 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.651838 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="981ec975-bd6c-4e89-9b03-b3e2339fe0da" containerName="payload-pre-processing" Sep 08 07:23:37.656129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.656102 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" Sep 08 07:23:37.658831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.658809 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 07:23:37.658949 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.658839 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-824nh\"" Sep 08 07:23:37.663334 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.663308 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4"] Sep 08 07:23:37.782131 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.782094 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c1828ad-a3d3-471e-8f2b-53a7d60a4a24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4\" (UID: \"8c1828ad-a3d3-471e-8f2b-53a7d60a4a24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" Sep 08 07:23:37.882809 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.882776 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c1828ad-a3d3-471e-8f2b-53a7d60a4a24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4\" (UID: \"8c1828ad-a3d3-471e-8f2b-53a7d60a4a24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" Sep 08 07:23:37.883437 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.883414 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c1828ad-a3d3-471e-8f2b-53a7d60a4a24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4\" (UID: \"8c1828ad-a3d3-471e-8f2b-53a7d60a4a24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" Sep 08 07:23:37.968523 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:37.968490 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" Sep 08 07:23:38.239677 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:38.239635 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4"] Sep 08 07:23:38.240968 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:23:38.240937 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c1828ad_a3d3_471e_8f2b_53a7d60a4a24.slice/crio-b5a0c9de50c0862771204fa64f5324c75b3fc3c20292479c0d9acee7482838ed WatchSource:0}: Error finding container b5a0c9de50c0862771204fa64f5324c75b3fc3c20292479c0d9acee7482838ed: Status 404 returned error can't find the container with id b5a0c9de50c0862771204fa64f5324c75b3fc3c20292479c0d9acee7482838ed Sep 08 07:23:38.861346 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:38.861246 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" event={"ID":"8c1828ad-a3d3-471e-8f2b-53a7d60a4a24","Type":"ContainerStarted","Data":"269c1bacf42eb5c1db61e27f0ac5cf9b2fbec3e33329d286dd0c5538402c6770"} Sep 08 07:23:38.861346 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:38.861322 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" event={"ID":"8c1828ad-a3d3-471e-8f2b-53a7d60a4a24","Type":"ContainerStarted","Data":"b5a0c9de50c0862771204fa64f5324c75b3fc3c20292479c0d9acee7482838ed"} Sep 08 07:23:38.878306 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:38.878258 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" podStartSLOduration=1.878242052 podStartE2EDuration="1.878242052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:23:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:23:38.876324513 +0000 UTC m=+1133.554287626" watchObservedRunningTime="2026-09-08 07:23:38.878242052 +0000 UTC m=+1133.556205169" Sep 08 07:23:39.868564 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:39.868532 2571 generic.go:356] "Generic (PLEG): container finished" podID="8c1828ad-a3d3-471e-8f2b-53a7d60a4a24" containerID="269c1bacf42eb5c1db61e27f0ac5cf9b2fbec3e33329d286dd0c5538402c6770" exitCode=0 Sep 08 07:23:39.869013 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:39.868573 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" event={"ID":"8c1828ad-a3d3-471e-8f2b-53a7d60a4a24","Type":"ContainerDied","Data":"269c1bacf42eb5c1db61e27f0ac5cf9b2fbec3e33329d286dd0c5538402c6770"} Sep 08 07:23:41.005466 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:41.005444 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" Sep 08 07:23:41.114963 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:41.114928 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c1828ad-a3d3-471e-8f2b-53a7d60a4a24-maas-api-service-ca\") pod \"8c1828ad-a3d3-471e-8f2b-53a7d60a4a24\" (UID: \"8c1828ad-a3d3-471e-8f2b-53a7d60a4a24\") " Sep 08 07:23:41.115299 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:41.115277 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c1828ad-a3d3-471e-8f2b-53a7d60a4a24-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c1828ad-a3d3-471e-8f2b-53a7d60a4a24" (UID: "8c1828ad-a3d3-471e-8f2b-53a7d60a4a24"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:23:41.216467 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:41.216437 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c1828ad-a3d3-471e-8f2b-53a7d60a4a24-maas-api-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:41.880104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:41.880073 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" Sep 08 07:23:41.880295 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:41.880105 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4" event={"ID":"8c1828ad-a3d3-471e-8f2b-53a7d60a4a24","Type":"ContainerDied","Data":"b5a0c9de50c0862771204fa64f5324c75b3fc3c20292479c0d9acee7482838ed"} Sep 08 07:23:41.880295 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:41.880153 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5a0c9de50c0862771204fa64f5324c75b3fc3c20292479c0d9acee7482838ed" Sep 08 07:23:42.592616 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.592557 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:23:42.731842 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.731807 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-podinfo\") pod \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " Sep 08 07:23:42.732031 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.731868 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w966b\" (UniqueName: \"kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-kube-api-access-w966b\") pod \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " Sep 08 07:23:42.732031 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.731920 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-envoy\") pod \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " Sep 08 07:23:42.732031 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.731956 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-token\") pod \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " Sep 08 07:23:42.732031 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.731997 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istiod-ca-cert\") pod \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " Sep 08 07:23:42.732283 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.732037 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-socket\") pod \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " Sep 08 07:23:42.732283 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.732070 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-credential-socket\") pod \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " Sep 08 07:23:42.732283 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.732265 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-certs\") pod \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " Sep 08 07:23:42.732439 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.732366 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-data\") pod \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\" (UID: \"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78\") " Sep 08 07:23:42.732933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.732624 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" (UID: "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:23:42.732933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.732675 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" (UID: "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:23:42.732933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.732838 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" (UID: "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:23:42.732933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.732766 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istiod-ca-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:42.732933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.732886 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:42.732933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.732907 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" (UID: "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:23:42.733258 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.733008 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" (UID: "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:23:42.734087 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.734064 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" (UID: "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:23:42.734277 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.734260 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" (UID: "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 07:23:42.734514 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.734496 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" (UID: "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:23:42.734563 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.734548 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-kube-api-access-w966b" (OuterVolumeSpecName: "kube-api-access-w966b") pod "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" (UID: "c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78"). InnerVolumeSpecName "kube-api-access-w966b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:23:42.833529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.833493 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-data\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:42.833529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.833522 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-podinfo\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:42.833529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.833532 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w966b\" (UniqueName: \"kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-kube-api-access-w966b\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:42.833529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.833540 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-envoy\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:42.833786 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.833551 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-istio-token\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:42.833786 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.833559 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-credential-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:42.833786 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.833567 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78-workload-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:23:42.887099 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.887021 2571 generic.go:356] "Generic (PLEG): container finished" podID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerID="1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0" exitCode=0 Sep 08 07:23:42.887099 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.887064 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" Sep 08 07:23:42.887271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.887098 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" event={"ID":"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78","Type":"ContainerDied","Data":"1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0"} Sep 08 07:23:42.887271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.887149 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7" event={"ID":"c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78","Type":"ContainerDied","Data":"40175539ef988743fd6835e09d36679bc8ec16410bf5e8e682b810403719cb97"} Sep 08 07:23:42.887271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.887172 2571 scope.go:117] "RemoveContainer" containerID="1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0" Sep 08 07:23:42.896588 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.896571 2571 scope.go:117] "RemoveContainer" containerID="4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10" Sep 08 07:23:42.904559 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.904542 2571 scope.go:117] "RemoveContainer" containerID="1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0" Sep 08 07:23:42.904813 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:23:42.904793 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0\": container with ID starting with 1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0 not found: ID does not exist" containerID="1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0" Sep 08 07:23:42.904882 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.904821 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0"} err="failed to get container status \"1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0\": rpc error: code = NotFound desc = could not find container \"1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0\": container with ID starting with 1506918d3d8c239ab6d8e3261f581c01cc2cc3a6fced1ef6a645ef7abe7d5de0 not found: ID does not exist" Sep 08 07:23:42.904882 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.904839 2571 scope.go:117] "RemoveContainer" containerID="4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10" Sep 08 07:23:42.905074 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:23:42.905056 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10\": container with ID starting with 4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10 not found: ID does not exist" containerID="4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10" Sep 08 07:23:42.905113 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.905081 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10"} err="failed to get container status \"4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10\": rpc error: code = NotFound desc = could not find container \"4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10\": container with ID starting with 4dabc319ea867e59500bd33d08de633c61934e266cf5c23b6c3dc466574dbe10 not found: ID does not exist" Sep 08 07:23:42.909568 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.909548 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7"] Sep 08 07:23:42.911769 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:42.911750 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-620112-openshift-default-664bd57b66-927l7"] Sep 08 07:23:43.808895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:43.808860 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" path="/var/lib/kubelet/pods/c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78/volumes" Sep 08 07:23:48.448703 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:48.448668 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4"] Sep 08 07:23:48.453395 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:48.453365 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6f5c4f-s6nk4"] Sep 08 07:23:49.810197 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:49.810163 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c1828ad-a3d3-471e-8f2b-53a7d60a4a24" path="/var/lib/kubelet/pods/8c1828ad-a3d3-471e-8f2b-53a7d60a4a24/volumes" Sep 08 07:23:55.537300 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:55.537265 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq"] Sep 08 07:23:55.537790 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:55.537521 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" containerID="cri-o://b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7" gracePeriod=30 Sep 08 07:23:57.576780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:57.576742 2571 patch_prober.go:28] interesting pod/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 07:23:57.577678 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:57.577372 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 07:23:58.227087 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.227055 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l"] Sep 08 07:23:58.227449 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.227432 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" Sep 08 07:23:58.227449 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.227448 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" Sep 08 07:23:58.227606 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.227460 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" Sep 08 07:23:58.227606 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.227465 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" Sep 08 07:23:58.227606 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.227488 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c1828ad-a3d3-471e-8f2b-53a7d60a4a24" containerName="revoke-keys" Sep 08 07:23:58.227606 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.227494 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c1828ad-a3d3-471e-8f2b-53a7d60a4a24" containerName="revoke-keys" Sep 08 07:23:58.227606 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.227548 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" Sep 08 07:23:58.227606 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.227562 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c1828ad-a3d3-471e-8f2b-53a7d60a4a24" containerName="revoke-keys" Sep 08 07:23:58.232180 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.232164 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:23:58.234707 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.234684 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-37abc6-dockercfg-n2vnr\"" Sep 08 07:23:58.234707 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.234704 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-37abc6\"" Sep 08 07:23:58.239683 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.239546 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l"] Sep 08 07:23:58.271675 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.271651 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qpvq\" (UniqueName: \"kubernetes.io/projected/dc378680-0ed1-4273-82f1-838ee35c5a9d-kube-api-access-7qpvq\") pod \"payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l\" (UID: \"dc378680-0ed1-4273-82f1-838ee35c5a9d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:23:58.271813 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.271716 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc378680-0ed1-4273-82f1-838ee35c5a9d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l\" (UID: \"dc378680-0ed1-4273-82f1-838ee35c5a9d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:23:58.373160 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.373121 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qpvq\" (UniqueName: \"kubernetes.io/projected/dc378680-0ed1-4273-82f1-838ee35c5a9d-kube-api-access-7qpvq\") pod \"payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l\" (UID: \"dc378680-0ed1-4273-82f1-838ee35c5a9d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:23:58.373345 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.373200 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc378680-0ed1-4273-82f1-838ee35c5a9d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l\" (UID: \"dc378680-0ed1-4273-82f1-838ee35c5a9d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:23:58.373843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.373821 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc378680-0ed1-4273-82f1-838ee35c5a9d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l\" (UID: \"dc378680-0ed1-4273-82f1-838ee35c5a9d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:23:58.381888 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.381861 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qpvq\" (UniqueName: \"kubernetes.io/projected/dc378680-0ed1-4273-82f1-838ee35c5a9d-kube-api-access-7qpvq\") pod \"payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l\" (UID: \"dc378680-0ed1-4273-82f1-838ee35c5a9d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:23:58.543133 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.543042 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:23:58.813837 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.813812 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l"] Sep 08 07:23:58.815035 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:23:58.815011 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc378680_0ed1_4273_82f1_838ee35c5a9d.slice/crio-577c1746b93b766016498a5ab3ab24f0d3faef8670a1dcf0aa573008cc18bd94 WatchSource:0}: Error finding container 577c1746b93b766016498a5ab3ab24f0d3faef8670a1dcf0aa573008cc18bd94: Status 404 returned error can't find the container with id 577c1746b93b766016498a5ab3ab24f0d3faef8670a1dcf0aa573008cc18bd94 Sep 08 07:23:58.981320 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.981280 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" event={"ID":"dc378680-0ed1-4273-82f1-838ee35c5a9d","Type":"ContainerStarted","Data":"eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c"} Sep 08 07:23:58.981489 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.981340 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" event={"ID":"dc378680-0ed1-4273-82f1-838ee35c5a9d","Type":"ContainerStarted","Data":"577c1746b93b766016498a5ab3ab24f0d3faef8670a1dcf0aa573008cc18bd94"} Sep 08 07:23:58.981489 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.981396 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:23:58.999849 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:23:58.999799 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" podStartSLOduration=0.999783426 podStartE2EDuration="999.783426ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:23:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:23:58.997768271 +0000 UTC m=+1153.675731388" watchObservedRunningTime="2026-09-08 07:23:58.999783426 +0000 UTC m=+1153.677746542" Sep 08 07:24:00.793953 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.793928 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:24:00.895732 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.895629 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-podinfo\") pod \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " Sep 08 07:24:00.895732 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.895684 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-envoy\") pod \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " Sep 08 07:24:00.896002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.895748 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istiod-ca-cert\") pod \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " Sep 08 07:24:00.896002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.895786 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-credential-socket\") pod \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " Sep 08 07:24:00.896002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.895815 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccclm\" (UniqueName: \"kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-kube-api-access-ccclm\") pod \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " Sep 08 07:24:00.896002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.895886 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-socket\") pod \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " Sep 08 07:24:00.896002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.895928 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-certs\") pod \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " Sep 08 07:24:00.896002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.895969 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-data\") pod \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " Sep 08 07:24:00.896334 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.896010 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-token\") pod \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\" (UID: \"1cd78c9a-8f85-4d85-8338-8012225a9a1d\") " Sep 08 07:24:00.896334 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.896171 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1cd78c9a-8f85-4d85-8338-8012225a9a1d" (UID: "1cd78c9a-8f85-4d85-8338-8012225a9a1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:24:00.896334 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.896286 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1cd78c9a-8f85-4d85-8338-8012225a9a1d" (UID: "1cd78c9a-8f85-4d85-8338-8012225a9a1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:24:00.896483 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.896445 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-data" (OuterVolumeSpecName: "istio-data") pod "1cd78c9a-8f85-4d85-8338-8012225a9a1d" (UID: "1cd78c9a-8f85-4d85-8338-8012225a9a1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:24:00.896992 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.896587 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-data\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:24:00.896992 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.896612 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:24:00.896992 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.896627 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-workload-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:24:00.896992 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.896656 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1cd78c9a-8f85-4d85-8338-8012225a9a1d" (UID: "1cd78c9a-8f85-4d85-8338-8012225a9a1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:24:00.896992 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.896156 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1cd78c9a-8f85-4d85-8338-8012225a9a1d" (UID: "1cd78c9a-8f85-4d85-8338-8012225a9a1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:24:00.898592 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.898562 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-kube-api-access-ccclm" (OuterVolumeSpecName: "kube-api-access-ccclm") pod "1cd78c9a-8f85-4d85-8338-8012225a9a1d" (UID: "1cd78c9a-8f85-4d85-8338-8012225a9a1d"). InnerVolumeSpecName "kube-api-access-ccclm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:24:00.898688 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.898626 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-token" (OuterVolumeSpecName: "istio-token") pod "1cd78c9a-8f85-4d85-8338-8012225a9a1d" (UID: "1cd78c9a-8f85-4d85-8338-8012225a9a1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:24:00.898765 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.898749 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1cd78c9a-8f85-4d85-8338-8012225a9a1d" (UID: "1cd78c9a-8f85-4d85-8338-8012225a9a1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 07:24:00.898820 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.898771 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1cd78c9a-8f85-4d85-8338-8012225a9a1d" (UID: "1cd78c9a-8f85-4d85-8338-8012225a9a1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:24:00.994388 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.994348 2571 generic.go:356] "Generic (PLEG): container finished" podID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerID="b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7" exitCode=0 Sep 08 07:24:00.994564 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.994400 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" Sep 08 07:24:00.994564 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.994425 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" event={"ID":"1cd78c9a-8f85-4d85-8338-8012225a9a1d","Type":"ContainerDied","Data":"b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7"} Sep 08 07:24:00.994564 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.994481 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq" event={"ID":"1cd78c9a-8f85-4d85-8338-8012225a9a1d","Type":"ContainerDied","Data":"bb6582439af94c62700f2009611fffb413c0ccbd79f099c719efcc0f824c8007"} Sep 08 07:24:00.994564 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.994502 2571 scope.go:117] "RemoveContainer" containerID="b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7" Sep 08 07:24:00.997108 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.997086 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-token\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:24:00.997108 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.997106 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-podinfo\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:24:00.997302 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.997115 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istio-envoy\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:24:00.997302 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.997123 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cd78c9a-8f85-4d85-8338-8012225a9a1d-istiod-ca-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:24:00.997302 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.997131 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cd78c9a-8f85-4d85-8338-8012225a9a1d-credential-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:24:00.997302 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:00.997144 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccclm\" (UniqueName: \"kubernetes.io/projected/1cd78c9a-8f85-4d85-8338-8012225a9a1d-kube-api-access-ccclm\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:24:01.004247 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.004205 2571 scope.go:117] "RemoveContainer" containerID="27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad" Sep 08 07:24:01.012476 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.012461 2571 scope.go:117] "RemoveContainer" containerID="b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7" Sep 08 07:24:01.012749 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:24:01.012729 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7\": container with ID starting with b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7 not found: ID does not exist" containerID="b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7" Sep 08 07:24:01.012807 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.012758 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7"} err="failed to get container status \"b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7\": rpc error: code = NotFound desc = could not find container \"b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7\": container with ID starting with b41be2e8b4b53ce8b7694b9d7fd6869330d62f0d1a72853d1eb550d1cc03b6b7 not found: ID does not exist" Sep 08 07:24:01.012807 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.012774 2571 scope.go:117] "RemoveContainer" containerID="27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad" Sep 08 07:24:01.013017 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:24:01.013002 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad\": container with ID starting with 27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad not found: ID does not exist" containerID="27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad" Sep 08 07:24:01.013061 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.013023 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad"} err="failed to get container status \"27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad\": rpc error: code = NotFound desc = could not find container \"27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad\": container with ID starting with 27042ecffd534bdca8388e51247b288e62ba023546b2d82acf13b0cc553886ad not found: ID does not exist" Sep 08 07:24:01.019120 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.019096 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq"] Sep 08 07:24:01.022502 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.022482 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-6f5c4f-openshift-default-7994f4f584-rxqgq"] Sep 08 07:24:01.226892 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.226859 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7"] Sep 08 07:24:01.227265 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.227252 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" Sep 08 07:24:01.227310 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.227266 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" Sep 08 07:24:01.227310 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.227274 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" Sep 08 07:24:01.227310 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.227280 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" Sep 08 07:24:01.227412 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.227330 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" Sep 08 07:24:01.227412 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.227340 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f4e2bc-59f2-4b2f-8f8b-5a772224ab78" containerName="istio-proxy" Sep 08 07:24:01.231930 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.231914 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:24:01.234777 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.234757 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-fb59c2-dockercfg-dng46\"" Sep 08 07:24:01.234903 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.234796 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-fb59c2\"" Sep 08 07:24:01.240808 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.240788 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7"] Sep 08 07:24:01.300486 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.300444 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p98np\" (UniqueName: \"kubernetes.io/projected/3b201b98-b85e-480f-bc41-0c8729533142-kube-api-access-p98np\") pod \"payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7\" (UID: \"3b201b98-b85e-480f-bc41-0c8729533142\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:24:01.300698 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.300503 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b201b98-b85e-480f-bc41-0c8729533142-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7\" (UID: \"3b201b98-b85e-480f-bc41-0c8729533142\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:24:01.401284 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.401251 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p98np\" (UniqueName: \"kubernetes.io/projected/3b201b98-b85e-480f-bc41-0c8729533142-kube-api-access-p98np\") pod \"payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7\" (UID: \"3b201b98-b85e-480f-bc41-0c8729533142\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:24:01.401284 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.401287 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b201b98-b85e-480f-bc41-0c8729533142-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7\" (UID: \"3b201b98-b85e-480f-bc41-0c8729533142\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:24:01.401986 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.401966 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b201b98-b85e-480f-bc41-0c8729533142-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7\" (UID: \"3b201b98-b85e-480f-bc41-0c8729533142\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:24:01.413817 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.413792 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p98np\" (UniqueName: \"kubernetes.io/projected/3b201b98-b85e-480f-bc41-0c8729533142-kube-api-access-p98np\") pod \"payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7\" (UID: \"3b201b98-b85e-480f-bc41-0c8729533142\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:24:01.543782 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.543695 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:24:01.809710 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.809634 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" path="/var/lib/kubelet/pods/1cd78c9a-8f85-4d85-8338-8012225a9a1d/volumes" Sep 08 07:24:01.817301 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:01.817026 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7"] Sep 08 07:24:01.819756 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:24:01.819727 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b201b98_b85e_480f_bc41_0c8729533142.slice/crio-8f64f68b2b62a9e11fb5d153aaf53205d0d5daecab70618719367b88867d1259 WatchSource:0}: Error finding container 8f64f68b2b62a9e11fb5d153aaf53205d0d5daecab70618719367b88867d1259: Status 404 returned error can't find the container with id 8f64f68b2b62a9e11fb5d153aaf53205d0d5daecab70618719367b88867d1259 Sep 08 07:24:02.003076 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:02.003040 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" event={"ID":"3b201b98-b85e-480f-bc41-0c8729533142","Type":"ContainerStarted","Data":"f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e"} Sep 08 07:24:02.003269 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:02.003099 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" event={"ID":"3b201b98-b85e-480f-bc41-0c8729533142","Type":"ContainerStarted","Data":"8f64f68b2b62a9e11fb5d153aaf53205d0d5daecab70618719367b88867d1259"} Sep 08 07:24:02.003269 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:02.003261 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:24:02.022016 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:02.021960 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" podStartSLOduration=1.021943896 podStartE2EDuration="1.021943896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:24:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:24:02.020056207 +0000 UTC m=+1156.698019348" watchObservedRunningTime="2026-09-08 07:24:02.021943896 +0000 UTC m=+1156.699907012" Sep 08 07:24:09.989142 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:09.989065 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:24:13.010792 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:13.010757 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:24:21.899702 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.899670 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v"] Sep 08 07:24:21.900113 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.900100 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cd78c9a-8f85-4d85-8338-8012225a9a1d" containerName="istio-proxy" Sep 08 07:24:21.907303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.905245 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:21.909007 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.908980 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-0ca807-openshift-default-dockercfg-jxzcx\"" Sep 08 07:24:21.914427 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.914402 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v"] Sep 08 07:24:21.997970 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.997933 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-envoy\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:21.997970 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.997968 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-token\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:21.998170 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.998001 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5v2p\" (UniqueName: \"kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-kube-api-access-r5v2p\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:21.998170 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.998088 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80809f81-3613-4de4-939a-3e32f6a1c87a-istiod-ca-cert\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:21.998170 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.998130 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-podinfo\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:21.998313 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.998237 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-certs\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:21.998313 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.998269 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-data\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:21.998313 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.998295 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-socket\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:21.998411 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:21.998336 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-credential-socket\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099061 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099017 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-certs\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099061 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099061 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-data\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099079 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-socket\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099101 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-credential-socket\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099141 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-envoy\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099167 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-token\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099194 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5v2p\" (UniqueName: \"kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-kube-api-access-r5v2p\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099266 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80809f81-3613-4de4-939a-3e32f6a1c87a-istiod-ca-cert\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099302 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-podinfo\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099700 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099516 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-socket\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099700 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099548 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-certs\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099700 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099578 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-data\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.099700 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.099638 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-credential-socket\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.100126 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.100103 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80809f81-3613-4de4-939a-3e32f6a1c87a-istiod-ca-cert\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.101786 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.101762 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-podinfo\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.102030 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.102010 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-envoy\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.109239 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.109154 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-token\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.115616 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.115584 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5v2p\" (UniqueName: \"kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-kube-api-access-r5v2p\") pod \"e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.220540 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.220501 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:22.507773 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.507743 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v"] Sep 08 07:24:22.509234 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:24:22.509183 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80809f81_3613_4de4_939a_3e32f6a1c87a.slice/crio-d7bc5f169609e0bee99605c2dd408c1d6ec703178253ae26e86dff4b70f3c7cc WatchSource:0}: Error finding container d7bc5f169609e0bee99605c2dd408c1d6ec703178253ae26e86dff4b70f3c7cc: Status 404 returned error can't find the container with id d7bc5f169609e0bee99605c2dd408c1d6ec703178253ae26e86dff4b70f3c7cc Sep 08 07:24:22.511416 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.511382 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:24:22.511523 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.511464 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:24:22.511523 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:22.511511 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:24:23.128310 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:23.128274 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" event={"ID":"80809f81-3613-4de4-939a-3e32f6a1c87a","Type":"ContainerStarted","Data":"e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607"} Sep 08 07:24:23.128696 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:23.128336 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" event={"ID":"80809f81-3613-4de4-939a-3e32f6a1c87a","Type":"ContainerStarted","Data":"d7bc5f169609e0bee99605c2dd408c1d6ec703178253ae26e86dff4b70f3c7cc"} Sep 08 07:24:23.147888 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:23.147836 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" podStartSLOduration=2.147818127 podStartE2EDuration="2.147818127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:24:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:24:23.147189084 +0000 UTC m=+1177.825152242" watchObservedRunningTime="2026-09-08 07:24:23.147818127 +0000 UTC m=+1177.825781245" Sep 08 07:24:23.221005 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:23.220968 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:23.226020 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:23.225994 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:24.133637 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:24.133604 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:24.134704 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:24.134684 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:29.469656 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.469619 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj"] Sep 08 07:24:29.473141 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.473124 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:24:29.475648 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.475624 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-217bc2\"" Sep 08 07:24:29.475812 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.475794 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-217bc2-dockercfg-hkmbn\"" Sep 08 07:24:29.480006 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.479958 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm"] Sep 08 07:24:29.483858 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.483838 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj"] Sep 08 07:24:29.483945 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.483923 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:24:29.495753 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.495726 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm"] Sep 08 07:24:29.572295 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.572259 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6blhr\" (UniqueName: \"kubernetes.io/projected/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-kube-api-access-6blhr\") pod \"payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm\" (UID: \"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:24:29.572475 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.572347 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4vlc\" (UniqueName: \"kubernetes.io/projected/74eb1128-c920-4f17-8e22-f335eb5d4010-kube-api-access-z4vlc\") pod \"payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj\" (UID: \"74eb1128-c920-4f17-8e22-f335eb5d4010\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:24:29.572475 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.572377 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74eb1128-c920-4f17-8e22-f335eb5d4010-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj\" (UID: \"74eb1128-c920-4f17-8e22-f335eb5d4010\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:24:29.572567 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.572476 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm\" (UID: \"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:24:29.673792 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.673753 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6blhr\" (UniqueName: \"kubernetes.io/projected/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-kube-api-access-6blhr\") pod \"payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm\" (UID: \"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:24:29.673997 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.673821 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4vlc\" (UniqueName: \"kubernetes.io/projected/74eb1128-c920-4f17-8e22-f335eb5d4010-kube-api-access-z4vlc\") pod \"payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj\" (UID: \"74eb1128-c920-4f17-8e22-f335eb5d4010\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:24:29.673997 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.673843 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74eb1128-c920-4f17-8e22-f335eb5d4010-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj\" (UID: \"74eb1128-c920-4f17-8e22-f335eb5d4010\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:24:29.673997 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.673923 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm\" (UID: \"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:24:29.674680 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.674662 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74eb1128-c920-4f17-8e22-f335eb5d4010-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj\" (UID: \"74eb1128-c920-4f17-8e22-f335eb5d4010\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:24:29.674755 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.674662 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm\" (UID: \"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:24:29.682593 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.682569 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4vlc\" (UniqueName: \"kubernetes.io/projected/74eb1128-c920-4f17-8e22-f335eb5d4010-kube-api-access-z4vlc\") pod \"payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj\" (UID: \"74eb1128-c920-4f17-8e22-f335eb5d4010\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:24:29.682707 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.682594 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6blhr\" (UniqueName: \"kubernetes.io/projected/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-kube-api-access-6blhr\") pod \"payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm\" (UID: \"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:24:29.785842 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.785763 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:24:29.794721 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:29.794696 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:24:30.064662 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:30.064637 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj"] Sep 08 07:24:30.065983 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:24:30.065949 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74eb1128_c920_4f17_8e22_f335eb5d4010.slice/crio-1507c85e9410de1c0daf1321ef44698723b50475be9a8a6fe8042cdd2ee4cd8d WatchSource:0}: Error finding container 1507c85e9410de1c0daf1321ef44698723b50475be9a8a6fe8042cdd2ee4cd8d: Status 404 returned error can't find the container with id 1507c85e9410de1c0daf1321ef44698723b50475be9a8a6fe8042cdd2ee4cd8d Sep 08 07:24:30.073360 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:24:30.073336 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03125a9d_ba1a_48af_93a9_2e8f7a6c9bd8.slice/crio-3ed9d89476874b999581939dc2a67776901edcd686bdec1fdd3b3ff88340ddeb WatchSource:0}: Error finding container 3ed9d89476874b999581939dc2a67776901edcd686bdec1fdd3b3ff88340ddeb: Status 404 returned error can't find the container with id 3ed9d89476874b999581939dc2a67776901edcd686bdec1fdd3b3ff88340ddeb Sep 08 07:24:30.073533 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:30.073509 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm"] Sep 08 07:24:30.174433 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:30.174395 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" event={"ID":"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8","Type":"ContainerStarted","Data":"8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45"} Sep 08 07:24:30.174640 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:30.174463 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" event={"ID":"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8","Type":"ContainerStarted","Data":"3ed9d89476874b999581939dc2a67776901edcd686bdec1fdd3b3ff88340ddeb"} Sep 08 07:24:30.174640 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:30.174603 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:24:30.176441 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:30.176407 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" event={"ID":"74eb1128-c920-4f17-8e22-f335eb5d4010","Type":"ContainerStarted","Data":"f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a"} Sep 08 07:24:30.176585 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:30.176456 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:24:30.176585 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:30.176482 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" event={"ID":"74eb1128-c920-4f17-8e22-f335eb5d4010","Type":"ContainerStarted","Data":"1507c85e9410de1c0daf1321ef44698723b50475be9a8a6fe8042cdd2ee4cd8d"} Sep 08 07:24:30.190648 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:30.190595 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" podStartSLOduration=1.190574726 podStartE2EDuration="1.190574726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:24:30.189847673 +0000 UTC m=+1184.867810802" watchObservedRunningTime="2026-09-08 07:24:30.190574726 +0000 UTC m=+1184.868537844" Sep 08 07:24:30.204734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:30.204677 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" podStartSLOduration=1.204658833 podStartE2EDuration="1.204658833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:24:30.204297781 +0000 UTC m=+1184.882260890" watchObservedRunningTime="2026-09-08 07:24:30.204658833 +0000 UTC m=+1184.882621950" Sep 08 07:24:41.184470 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:41.184435 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:24:41.184840 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:41.184529 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:24:44.268042 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:44.268007 2571 generic.go:356] "Generic (PLEG): container finished" podID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerID="e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607" exitCode=0 Sep 08 07:24:44.268042 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:44.268046 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" event={"ID":"80809f81-3613-4de4-939a-3e32f6a1c87a","Type":"ContainerDied","Data":"e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607"} Sep 08 07:24:44.268520 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:44.268477 2571 scope.go:117] "RemoveContainer" containerID="e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607" Sep 08 07:24:44.272498 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:44.272450 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:24:44.272649 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:44.272541 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:24:44.272649 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:44.272576 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:24:45.276875 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:45.276835 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" event={"ID":"80809f81-3613-4de4-939a-3e32f6a1c87a","Type":"ContainerStarted","Data":"390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d"} Sep 08 07:24:46.220826 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:46.220792 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:46.221041 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:46.221014 2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 08 07:24:46.221098 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:46.221077 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 08 07:24:47.221826 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:47.221791 2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 08 07:24:47.222259 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:47.221848 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 08 07:24:48.220877 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:48.220839 2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 08 07:24:48.221080 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:48.220903 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 08 07:24:49.221263 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:49.221199 2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 08 07:24:49.221653 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:49.221291 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 08 07:24:50.221131 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:50.221092 2571 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 08 07:24:50.221336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:50.221154 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 08 07:24:51.224808 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:51.224775 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:51.225189 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:51.225045 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:24:51.225690 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:24:51.225672 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:25:07.548003 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.547968 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l"] Sep 08 07:25:07.551328 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.551305 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.553674 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.553653 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-15977b5c-openshift-default-dockercfg-9m74q\"" Sep 08 07:25:07.563064 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.563042 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l"] Sep 08 07:25:07.645551 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.645516 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6knl\" (UniqueName: \"kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-kube-api-access-v6knl\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.645720 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.645580 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-credential-socket\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.645720 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.645609 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-data\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.645720 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.645692 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istiod-ca-cert\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.645864 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.645774 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-socket\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.645864 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.645793 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-certs\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.645864 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.645815 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-envoy\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.645864 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.645847 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-podinfo\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.646008 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.645888 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-token\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.747446 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.747412 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istiod-ca-cert\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.747633 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.747484 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-socket\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.747633 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.747506 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-certs\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.747633 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.747523 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-envoy\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.747633 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.747544 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-podinfo\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.747633 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.747558 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-token\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.747633 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.747581 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6knl\" (UniqueName: \"kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-kube-api-access-v6knl\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.747949 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.747633 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-credential-socket\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.747949 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.747661 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-data\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.748104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.748068 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-credential-socket\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.748191 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.748114 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-socket\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.748191 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.748178 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-certs\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.748314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.748294 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istiod-ca-cert\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.748364 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.748312 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-data\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.749880 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.749864 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-envoy\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.750104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.750089 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-podinfo\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.756839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.756810 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-token\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.757084 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.757064 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6knl\" (UniqueName: \"kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-kube-api-access-v6knl\") pod \"e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:07.862883 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:07.862802 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:08.137423 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:08.137394 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l"] Sep 08 07:25:08.138325 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:25:08.138303 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ec2bcf_728c_46d3_bb15_27424f73ab7e.slice/crio-9c4bc7ea8b6108dd76cdb2e8e429a1edb70e2b23fecf11de87e16a547bfb4384 WatchSource:0}: Error finding container 9c4bc7ea8b6108dd76cdb2e8e429a1edb70e2b23fecf11de87e16a547bfb4384: Status 404 returned error can't find the container with id 9c4bc7ea8b6108dd76cdb2e8e429a1edb70e2b23fecf11de87e16a547bfb4384 Sep 08 07:25:08.140621 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:08.140583 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:25:08.140721 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:08.140655 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:25:08.140721 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:08.140683 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:25:08.421066 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:08.420980 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" event={"ID":"93ec2bcf-728c-46d3-bb15-27424f73ab7e","Type":"ContainerStarted","Data":"68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1"} Sep 08 07:25:08.421066 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:08.421032 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" event={"ID":"93ec2bcf-728c-46d3-bb15-27424f73ab7e","Type":"ContainerStarted","Data":"9c4bc7ea8b6108dd76cdb2e8e429a1edb70e2b23fecf11de87e16a547bfb4384"} Sep 08 07:25:08.443624 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:08.443572 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" podStartSLOduration=1.443557386 podStartE2EDuration="1.443557386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:25:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:25:08.441868438 +0000 UTC m=+1223.119831580" watchObservedRunningTime="2026-09-08 07:25:08.443557386 +0000 UTC m=+1223.121520501" Sep 08 07:25:08.863861 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:08.863827 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:09.868294 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:09.868264 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:10.432296 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:10.432261 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:10.433167 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:10.433148 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:29.550620 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:29.550519 2571 generic.go:356] "Generic (PLEG): container finished" podID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerID="68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1" exitCode=0 Sep 08 07:25:29.550620 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:29.550593 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" event={"ID":"93ec2bcf-728c-46d3-bb15-27424f73ab7e","Type":"ContainerDied","Data":"68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1"} Sep 08 07:25:29.551042 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:29.550968 2571 scope.go:117] "RemoveContainer" containerID="68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1" Sep 08 07:25:29.552110 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:29.552080 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:25:29.552247 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:29.552145 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:25:29.552247 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:29.552175 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:25:30.557944 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:30.557907 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" event={"ID":"93ec2bcf-728c-46d3-bb15-27424f73ab7e","Type":"ContainerStarted","Data":"ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa"} Sep 08 07:25:30.863958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:30.863873 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:30.864105 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:30.864061 2571 patch_prober.go:28] interesting pod/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 08 07:25:30.864146 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:30.864129 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 08 07:25:31.863332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:31.863293 2571 patch_prober.go:28] interesting pod/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 08 07:25:31.863718 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:31.863391 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 08 07:25:32.864109 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:32.864071 2571 patch_prober.go:28] interesting pod/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 08 07:25:32.864615 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:32.864146 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 08 07:25:33.864227 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:33.864171 2571 patch_prober.go:28] interesting pod/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 08 07:25:33.864618 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:33.864254 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 08 07:25:34.863254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:34.863191 2571 patch_prober.go:28] interesting pod/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 08 07:25:34.863445 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:34.863272 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 08 07:25:35.863238 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:35.863178 2571 patch_prober.go:28] interesting pod/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 08 07:25:35.863612 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:35.863255 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 08 07:25:36.867306 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:36.867274 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:36.867692 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:36.867561 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:36.868936 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:36.868916 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:25:45.914086 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:45.914054 2571 scope.go:117] "RemoveContainer" containerID="18bb09225407fd2c0acc39e69ca9087a9a3eb86183ab9b51e0e6cb7f6b1f012a" Sep 08 07:25:59.338031 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.337983 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx"] Sep 08 07:25:59.341864 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.341842 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.344534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.344511 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-15977b5c\"/\"openshift-service-ca.crt\"" Sep 08 07:25:59.345443 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.345424 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-15977b5c\"/\"default-dockercfg-mbwn6\"" Sep 08 07:25:59.345549 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.345430 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-15977b5c\"/\"kube-root-ca.crt\"" Sep 08 07:25:59.345549 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.345492 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-15977b5c\"/\"e2e-lifecycle-model-15977b5c-kserve-self-signed-certs\"" Sep 08 07:25:59.351839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.351813 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx"] Sep 08 07:25:59.459086 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.459050 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-dshm\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.459086 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.459093 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-home\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.459360 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.459133 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tmp-dir\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.459360 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.459193 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-model-cache\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.459360 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.459241 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tls-certs\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.459360 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.459276 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bjk6\" (UniqueName: \"kubernetes.io/projected/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-kube-api-access-4bjk6\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.559822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.559792 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-home\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.560001 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.559854 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tmp-dir\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.560001 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.559969 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-model-cache\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.560118 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.560000 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tls-certs\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.560118 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.560054 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bjk6\" (UniqueName: \"kubernetes.io/projected/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-kube-api-access-4bjk6\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.560252 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.560145 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-dshm\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.560252 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.560152 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tmp-dir\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.560374 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.560267 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-home\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.560432 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.560391 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-model-cache\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.562446 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.562415 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-dshm\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.562739 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.562715 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tls-certs\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.568941 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.568919 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bjk6\" (UniqueName: \"kubernetes.io/projected/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-kube-api-access-4bjk6\") pod \"e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.652790 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.652703 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:25:59.924863 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:25:59.924831 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx"] Sep 08 07:25:59.926402 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:25:59.926373 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4d94003_3ef9_4dcd_ae73_a6ff24651c86.slice/crio-6bbad59b5ae8a311bbee3ed0bb001d8b6ccf97366d486229ca19364cc1aa1172 WatchSource:0}: Error finding container 6bbad59b5ae8a311bbee3ed0bb001d8b6ccf97366d486229ca19364cc1aa1172: Status 404 returned error can't find the container with id 6bbad59b5ae8a311bbee3ed0bb001d8b6ccf97366d486229ca19364cc1aa1172 Sep 08 07:26:00.742205 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:26:00.742170 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" event={"ID":"f4d94003-3ef9-4dcd-ae73-a6ff24651c86","Type":"ContainerStarted","Data":"62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d"} Sep 08 07:26:00.742613 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:26:00.742232 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" event={"ID":"f4d94003-3ef9-4dcd-ae73-a6ff24651c86","Type":"ContainerStarted","Data":"6bbad59b5ae8a311bbee3ed0bb001d8b6ccf97366d486229ca19364cc1aa1172"} Sep 08 07:26:00.760415 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:26:00.760360 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" podStartSLOduration=1.760344349 podStartE2EDuration="1.760344349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:25:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:26:00.758451566 +0000 UTC m=+1275.436414682" watchObservedRunningTime="2026-09-08 07:26:00.760344349 +0000 UTC m=+1275.438307464" Sep 08 07:26:09.653437 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:26:09.653392 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:26:09.653876 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:26:09.653706 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:26:09.666540 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:26:09.666510 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:26:09.809841 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:26:09.809811 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:26:45.935472 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:26:45.935434 2571 scope.go:117] "RemoveContainer" containerID="d7c5280d8b47d0524d4113a5bb1d41fcc6a34549bfa14cfaeb1ba1625e904b3a" Sep 08 07:26:53.530043 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:26:53.530005 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx"] Sep 08 07:27:08.550100 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.550001 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" podUID="f4d94003-3ef9-4dcd-ae73-a6ff24651c86" containerName="main" containerID="cri-o://62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d" gracePeriod=45 Sep 08 07:27:08.796935 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.796910 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:27:08.934125 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934044 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tls-certs\") pod \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " Sep 08 07:27:08.934303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934140 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bjk6\" (UniqueName: \"kubernetes.io/projected/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-kube-api-access-4bjk6\") pod \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " Sep 08 07:27:08.934303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934177 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-dshm\") pod \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " Sep 08 07:27:08.934303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934255 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tmp-dir\") pod \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " Sep 08 07:27:08.934303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934281 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-home\") pod \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " Sep 08 07:27:08.934303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934299 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-model-cache\") pod \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\" (UID: \"f4d94003-3ef9-4dcd-ae73-a6ff24651c86\") " Sep 08 07:27:08.934575 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934494 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f4d94003-3ef9-4dcd-ae73-a6ff24651c86" (UID: "f4d94003-3ef9-4dcd-ae73-a6ff24651c86"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:27:08.934575 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934554 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-home" (OuterVolumeSpecName: "home") pod "f4d94003-3ef9-4dcd-ae73-a6ff24651c86" (UID: "f4d94003-3ef9-4dcd-ae73-a6ff24651c86"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:27:08.934716 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934699 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tmp-dir\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:08.934782 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934739 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-model-cache" (OuterVolumeSpecName: "model-cache") pod "f4d94003-3ef9-4dcd-ae73-a6ff24651c86" (UID: "f4d94003-3ef9-4dcd-ae73-a6ff24651c86"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:27:08.934782 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.934748 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-home\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:08.936393 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.936369 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-dshm" (OuterVolumeSpecName: "dshm") pod "f4d94003-3ef9-4dcd-ae73-a6ff24651c86" (UID: "f4d94003-3ef9-4dcd-ae73-a6ff24651c86"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:27:08.936393 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.936379 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f4d94003-3ef9-4dcd-ae73-a6ff24651c86" (UID: "f4d94003-3ef9-4dcd-ae73-a6ff24651c86"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:27:08.936505 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:08.936426 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-kube-api-access-4bjk6" (OuterVolumeSpecName: "kube-api-access-4bjk6") pod "f4d94003-3ef9-4dcd-ae73-a6ff24651c86" (UID: "f4d94003-3ef9-4dcd-ae73-a6ff24651c86"). InnerVolumeSpecName "kube-api-access-4bjk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:27:09.036134 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.036095 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bjk6\" (UniqueName: \"kubernetes.io/projected/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-kube-api-access-4bjk6\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:09.036134 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.036127 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-dshm\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:09.036134 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.036137 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-model-cache\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:09.036396 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.036146 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4d94003-3ef9-4dcd-ae73-a6ff24651c86-tls-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:09.137855 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.137820 2571 generic.go:356] "Generic (PLEG): container finished" podID="f4d94003-3ef9-4dcd-ae73-a6ff24651c86" containerID="62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d" exitCode=0 Sep 08 07:27:09.138027 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.137869 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" event={"ID":"f4d94003-3ef9-4dcd-ae73-a6ff24651c86","Type":"ContainerDied","Data":"62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d"} Sep 08 07:27:09.138027 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.137896 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" event={"ID":"f4d94003-3ef9-4dcd-ae73-a6ff24651c86","Type":"ContainerDied","Data":"6bbad59b5ae8a311bbee3ed0bb001d8b6ccf97366d486229ca19364cc1aa1172"} Sep 08 07:27:09.138027 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.137912 2571 scope.go:117] "RemoveContainer" containerID="62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d" Sep 08 07:27:09.138027 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.137861 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx" Sep 08 07:27:09.147285 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.147270 2571 scope.go:117] "RemoveContainer" containerID="62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d" Sep 08 07:27:09.147533 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:27:09.147516 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d\": container with ID starting with 62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d not found: ID does not exist" containerID="62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d" Sep 08 07:27:09.147583 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.147541 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d"} err="failed to get container status \"62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d\": rpc error: code = NotFound desc = could not find container \"62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d\": container with ID starting with 62b9b41d8a7a9cf592493e6e853ff4ab9614a0222752a66b29bf0e08d1c8b27d not found: ID does not exist" Sep 08 07:27:09.159932 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.159909 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx"] Sep 08 07:27:09.166387 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.166367 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-15977b5c/e2e-lifecycle-model-15977b5c-kserve-5c6755cff-kz7cx"] Sep 08 07:27:09.809329 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:09.809298 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4d94003-3ef9-4dcd-ae73-a6ff24651c86" path="/var/lib/kubelet/pods/f4d94003-3ef9-4dcd-ae73-a6ff24651c86/volumes" Sep 08 07:27:36.869571 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:36.869535 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l"] Sep 08 07:27:36.870155 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:36.869793 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" containerID="cri-o://ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa" gracePeriod=30 Sep 08 07:27:42.119697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.119672 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:27:42.245065 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245031 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-credential-socket\") pod \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " Sep 08 07:27:42.245351 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245127 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-socket\") pod \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " Sep 08 07:27:42.245351 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245148 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-envoy\") pod \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " Sep 08 07:27:42.245351 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245180 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-certs\") pod \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " Sep 08 07:27:42.245351 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245237 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6knl\" (UniqueName: \"kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-kube-api-access-v6knl\") pod \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " Sep 08 07:27:42.245351 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245271 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istiod-ca-cert\") pod \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " Sep 08 07:27:42.245351 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245306 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-data\") pod \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " Sep 08 07:27:42.245351 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245328 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-podinfo\") pod \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " Sep 08 07:27:42.245691 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245353 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-token\") pod \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\" (UID: \"93ec2bcf-728c-46d3-bb15-27424f73ab7e\") " Sep 08 07:27:42.245691 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245356 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93ec2bcf-728c-46d3-bb15-27424f73ab7e" (UID: "93ec2bcf-728c-46d3-bb15-27424f73ab7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:27:42.245691 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245423 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93ec2bcf-728c-46d3-bb15-27424f73ab7e" (UID: "93ec2bcf-728c-46d3-bb15-27424f73ab7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:27:42.245691 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245605 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93ec2bcf-728c-46d3-bb15-27424f73ab7e" (UID: "93ec2bcf-728c-46d3-bb15-27424f73ab7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:27:42.245895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245700 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-data" (OuterVolumeSpecName: "istio-data") pod "93ec2bcf-728c-46d3-bb15-27424f73ab7e" (UID: "93ec2bcf-728c-46d3-bb15-27424f73ab7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:27:42.245895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245716 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-credential-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:42.245895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245736 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:42.245895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245749 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-workload-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:42.245895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.245817 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93ec2bcf-728c-46d3-bb15-27424f73ab7e" (UID: "93ec2bcf-728c-46d3-bb15-27424f73ab7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:27:42.247445 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.247417 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93ec2bcf-728c-46d3-bb15-27424f73ab7e" (UID: "93ec2bcf-728c-46d3-bb15-27424f73ab7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:27:42.247636 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.247608 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-token" (OuterVolumeSpecName: "istio-token") pod "93ec2bcf-728c-46d3-bb15-27424f73ab7e" (UID: "93ec2bcf-728c-46d3-bb15-27424f73ab7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:27:42.247933 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.247905 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93ec2bcf-728c-46d3-bb15-27424f73ab7e" (UID: "93ec2bcf-728c-46d3-bb15-27424f73ab7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 07:27:42.248015 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.247933 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-kube-api-access-v6knl" (OuterVolumeSpecName: "kube-api-access-v6knl") pod "93ec2bcf-728c-46d3-bb15-27424f73ab7e" (UID: "93ec2bcf-728c-46d3-bb15-27424f73ab7e"). InnerVolumeSpecName "kube-api-access-v6knl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:27:42.346490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346455 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-envoy\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:42.346490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346490 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6knl\" (UniqueName: \"kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-kube-api-access-v6knl\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:42.346490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346504 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istiod-ca-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:42.346764 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346516 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-data\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:42.346764 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346527 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-podinfo\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:42.346764 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346538 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ec2bcf-728c-46d3-bb15-27424f73ab7e-istio-token\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:27:42.346870 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346779 2571 generic.go:356] "Generic (PLEG): container finished" podID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerID="ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa" exitCode=0 Sep 08 07:27:42.346870 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346847 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" Sep 08 07:27:42.346870 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346857 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" event={"ID":"93ec2bcf-728c-46d3-bb15-27424f73ab7e","Type":"ContainerDied","Data":"ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa"} Sep 08 07:27:42.346979 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346933 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l" event={"ID":"93ec2bcf-728c-46d3-bb15-27424f73ab7e","Type":"ContainerDied","Data":"9c4bc7ea8b6108dd76cdb2e8e429a1edb70e2b23fecf11de87e16a547bfb4384"} Sep 08 07:27:42.346979 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.346954 2571 scope.go:117] "RemoveContainer" containerID="ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa" Sep 08 07:27:42.356534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.356516 2571 scope.go:117] "RemoveContainer" containerID="68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1" Sep 08 07:27:42.364828 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.364811 2571 scope.go:117] "RemoveContainer" containerID="ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa" Sep 08 07:27:42.365089 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:27:42.365069 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa\": container with ID starting with ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa not found: ID does not exist" containerID="ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa" Sep 08 07:27:42.365148 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.365098 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa"} err="failed to get container status \"ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa\": rpc error: code = NotFound desc = could not find container \"ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa\": container with ID starting with ef9938491a2ce252a0950680f6e3a55ae51f6e5a69b64dfec0785b7d4eae9dfa not found: ID does not exist" Sep 08 07:27:42.365148 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.365114 2571 scope.go:117] "RemoveContainer" containerID="68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1" Sep 08 07:27:42.365352 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:27:42.365335 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1\": container with ID starting with 68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1 not found: ID does not exist" containerID="68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1" Sep 08 07:27:42.365402 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.365358 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1"} err="failed to get container status \"68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1\": rpc error: code = NotFound desc = could not find container \"68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1\": container with ID starting with 68c3a2f32e20cc9e90502e367a13030c6a885bfc1fcff572eed0531c6289d5d1 not found: ID does not exist" Sep 08 07:27:42.370008 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.369986 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l"] Sep 08 07:27:42.376458 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:42.376437 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-15977b5c-openshift-default-6f4b57f7dd-rsq6l"] Sep 08 07:27:43.809132 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:43.809088 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" path="/var/lib/kubelet/pods/93ec2bcf-728c-46d3-bb15-27424f73ab7e/volumes" Sep 08 07:27:45.957855 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:27:45.957821 2571 scope.go:117] "RemoveContainer" containerID="2c49922e4d131cc7efca9ef945b2fb691f04767604561222e3aca4082d8aeb78" Sep 08 07:28:31.198508 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.198474 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h"] Sep 08 07:28:31.198971 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.198952 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" Sep 08 07:28:31.199015 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.198975 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" Sep 08 07:28:31.199051 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.199017 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d94003-3ef9-4dcd-ae73-a6ff24651c86" containerName="main" Sep 08 07:28:31.199051 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.199027 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d94003-3ef9-4dcd-ae73-a6ff24651c86" containerName="main" Sep 08 07:28:31.199142 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.199128 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d94003-3ef9-4dcd-ae73-a6ff24651c86" containerName="main" Sep 08 07:28:31.199177 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.199146 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" Sep 08 07:28:31.199177 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.199158 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" Sep 08 07:28:31.202310 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.202291 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.204847 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.204821 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-7e01ea-openshift-default-dockercfg-95mxc\"" Sep 08 07:28:31.213262 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.213236 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h"] Sep 08 07:28:31.308710 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.308672 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-envoy\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.308710 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.308711 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-token\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.308972 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.308744 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istiod-ca-cert\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.308972 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.308844 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-data\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.308972 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.308927 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-credential-socket\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.309129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.308981 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-socket\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.309129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.309004 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-certs\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.309129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.309078 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-podinfo\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.309129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.309103 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s482\" (UniqueName: \"kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-kube-api-access-8s482\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.410105 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410062 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istiod-ca-cert\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.410344 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410139 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-data\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.410670 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410646 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-data\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.410778 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410681 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-credential-socket\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.410778 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410573 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-credential-socket\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.410892 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410780 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-socket\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.410892 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410809 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-certs\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.410994 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410891 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-podinfo\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.410994 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410916 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s482\" (UniqueName: \"kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-kube-api-access-8s482\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.410994 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410925 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istiod-ca-cert\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.411151 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.410996 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-envoy\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.411151 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.411024 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-token\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.411151 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.411060 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-socket\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.411381 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.411365 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-certs\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.413828 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.413799 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-envoy\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.414150 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.414128 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-podinfo\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.419343 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.419312 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-token\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.419467 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.419451 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s482\" (UniqueName: \"kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-kube-api-access-8s482\") pod \"e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.514345 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.514249 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:31.797017 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.796982 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h"] Sep 08 07:28:31.800440 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:28:31.800406 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71266cd5_c7bd_40a7_870b_effcdd2f27aa.slice/crio-94596e5443ba4888355fff5a0cadf7f53b6b5d7e93150aa5093a84f723c1e70e WatchSource:0}: Error finding container 94596e5443ba4888355fff5a0cadf7f53b6b5d7e93150aa5093a84f723c1e70e: Status 404 returned error can't find the container with id 94596e5443ba4888355fff5a0cadf7f53b6b5d7e93150aa5093a84f723c1e70e Sep 08 07:28:31.802604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.802564 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:28:31.802690 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.802637 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:28:31.802690 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:31.802669 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:28:32.628729 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:32.628695 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" event={"ID":"71266cd5-c7bd-40a7-870b-effcdd2f27aa","Type":"ContainerStarted","Data":"bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe"} Sep 08 07:28:32.629099 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:32.628773 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" event={"ID":"71266cd5-c7bd-40a7-870b-effcdd2f27aa","Type":"ContainerStarted","Data":"94596e5443ba4888355fff5a0cadf7f53b6b5d7e93150aa5093a84f723c1e70e"} Sep 08 07:28:32.650580 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:32.650527 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" podStartSLOduration=1.6505101660000001 podStartE2EDuration="1.650510166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:28:32.647832462 +0000 UTC m=+1427.325795581" watchObservedRunningTime="2026-09-08 07:28:32.650510166 +0000 UTC m=+1427.328473282" Sep 08 07:28:33.515388 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:33.515350 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:33.520875 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:33.520848 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:33.634236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:33.634182 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:33.635278 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:33.635254 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:45.992066 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:45.992028 2571 scope.go:117] "RemoveContainer" containerID="ce63f355b455ba5e72be52a35e9354a92f9904172595ff4480b77a475cd8e538" Sep 08 07:28:52.756138 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:52.756043 2571 generic.go:356] "Generic (PLEG): container finished" podID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerID="bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe" exitCode=0 Sep 08 07:28:52.756138 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:52.756120 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" event={"ID":"71266cd5-c7bd-40a7-870b-effcdd2f27aa","Type":"ContainerDied","Data":"bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe"} Sep 08 07:28:52.756613 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:52.756597 2571 scope.go:117] "RemoveContainer" containerID="bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe" Sep 08 07:28:52.757858 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:52.757822 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:28:52.757964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:52.757896 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:28:52.757964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:52.757928 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:28:53.764000 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:53.763965 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" event={"ID":"71266cd5-c7bd-40a7-870b-effcdd2f27aa","Type":"ContainerStarted","Data":"0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538"} Sep 08 07:28:54.514694 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:54.514650 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:28:54.514913 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:54.514892 2571 patch_prober.go:28] interesting pod/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 07:28:54.514960 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:54.514939 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 07:28:55.515307 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:55.515269 2571 patch_prober.go:28] interesting pod/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 07:28:55.515773 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:55.515335 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 07:28:56.515595 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:56.515555 2571 patch_prober.go:28] interesting pod/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 07:28:56.516006 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:56.515611 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 07:28:57.514826 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:57.514788 2571 patch_prober.go:28] interesting pod/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 07:28:57.515013 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:57.514857 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 07:28:58.515271 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:58.515200 2571 patch_prober.go:28] interesting pod/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 07:28:58.515669 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:58.515306 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 07:28:59.514931 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:59.514892 2571 patch_prober.go:28] interesting pod/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Sep 08 07:28:59.515102 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:28:59.514949 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Sep 08 07:29:00.161005 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.160969 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc"] Sep 08 07:29:00.161522 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.161507 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" Sep 08 07:29:00.161595 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.161525 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ec2bcf-728c-46d3-bb15-27424f73ab7e" containerName="istio-proxy" Sep 08 07:29:00.163891 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.163869 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.166738 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.166716 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f0e0d8-openshift-default-dockercfg-tb7p9\"" Sep 08 07:29:00.175998 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.175571 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc"] Sep 08 07:29:00.322970 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.322934 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-socket\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.323164 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.322976 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr9tl\" (UniqueName: \"kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-kube-api-access-pr9tl\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.323164 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.323005 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-envoy\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.323164 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.323063 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-token\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.323164 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.323104 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-data\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.323164 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.323128 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-podinfo\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.323164 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.323146 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-credential-socket\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.323402 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.323169 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istiod-ca-cert\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.323402 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.323284 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-certs\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.424702 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.424608 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-token\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.424702 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.424672 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-data\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.424702 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.424694 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-podinfo\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.424993 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.424716 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-credential-socket\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.424993 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.424734 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istiod-ca-cert\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.424993 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.424850 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-certs\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.424993 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.424899 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-socket\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.425178 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.425019 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr9tl\" (UniqueName: \"kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-kube-api-access-pr9tl\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.425178 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.425088 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-envoy\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.425318 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.425282 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-data\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.425377 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.425356 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-socket\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.425497 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.425425 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-credential-socket\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.425497 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.425461 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istiod-ca-cert\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.425497 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.425476 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-certs\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.427254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.427227 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-envoy\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.427357 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.427261 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-podinfo\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.434875 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.434853 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-token\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.435908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.435888 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr9tl\" (UniqueName: \"kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-kube-api-access-pr9tl\") pod \"e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.478756 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.478716 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:00.520391 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.520359 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:29:00.520790 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.520767 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:29:00.521552 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.521525 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:29:00.755576 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.755544 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc"] Sep 08 07:29:00.756567 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:29:00.756536 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b272ad8_0053_4aed_b6d3_4c160a1ca87a.slice/crio-5b68e285a40ba86ca92b6de6db2b559056a8feeea7bebcf02e326ee9fbc778af WatchSource:0}: Error finding container 5b68e285a40ba86ca92b6de6db2b559056a8feeea7bebcf02e326ee9fbc778af: Status 404 returned error can't find the container with id 5b68e285a40ba86ca92b6de6db2b559056a8feeea7bebcf02e326ee9fbc778af Sep 08 07:29:00.758850 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.758813 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:29:00.758958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.758893 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:29:00.758958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.758940 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:29:00.818698 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:00.818672 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" event={"ID":"8b272ad8-0053-4aed-b6d3-4c160a1ca87a","Type":"ContainerStarted","Data":"5b68e285a40ba86ca92b6de6db2b559056a8feeea7bebcf02e326ee9fbc778af"} Sep 08 07:29:01.825695 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:01.825658 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" event={"ID":"8b272ad8-0053-4aed-b6d3-4c160a1ca87a","Type":"ContainerStarted","Data":"b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147"} Sep 08 07:29:01.845908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:01.845851 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" podStartSLOduration=1.845834756 podStartE2EDuration="1.845834756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:29:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:29:01.845179187 +0000 UTC m=+1456.523142329" watchObservedRunningTime="2026-09-08 07:29:01.845834756 +0000 UTC m=+1456.523797873" Sep 08 07:29:02.479582 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:02.479546 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:02.484866 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:02.484838 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:02.831847 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:02.831761 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:02.832823 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:02.832802 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:05.467280 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.467245 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb"] Sep 08 07:29:05.470107 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.470091 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:29:05.472745 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.472719 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-f0e0d8-dockercfg-kcw9d\"" Sep 08 07:29:05.472745 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.472735 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f0e0d8\"" Sep 08 07:29:05.480805 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.480778 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb"] Sep 08 07:29:05.509815 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.509789 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh"] Sep 08 07:29:05.512496 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.512475 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:29:05.526730 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.526705 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh"] Sep 08 07:29:05.582920 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.582876 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb\" (UID: \"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:29:05.583116 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.582943 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg9pr\" (UniqueName: \"kubernetes.io/projected/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-kube-api-access-mg9pr\") pod \"payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb\" (UID: \"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:29:05.684288 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.684253 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh\" (UID: \"0ccf23c2-7966-4d79-9865-d4ce4e485c6d\") " pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:29:05.684471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.684311 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5z79\" (UniqueName: \"kubernetes.io/projected/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-kube-api-access-t5z79\") pod \"payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh\" (UID: \"0ccf23c2-7966-4d79-9865-d4ce4e485c6d\") " pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:29:05.684471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.684420 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb\" (UID: \"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:29:05.684471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.684458 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg9pr\" (UniqueName: \"kubernetes.io/projected/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-kube-api-access-mg9pr\") pod \"payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb\" (UID: \"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:29:05.685149 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.685128 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb\" (UID: \"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:29:05.692877 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.692852 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg9pr\" (UniqueName: \"kubernetes.io/projected/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-kube-api-access-mg9pr\") pod \"payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb\" (UID: \"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:29:05.781673 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.781567 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:29:05.785147 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.785118 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh\" (UID: \"0ccf23c2-7966-4d79-9865-d4ce4e485c6d\") " pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:29:05.785267 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.785176 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5z79\" (UniqueName: \"kubernetes.io/projected/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-kube-api-access-t5z79\") pod \"payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh\" (UID: \"0ccf23c2-7966-4d79-9865-d4ce4e485c6d\") " pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:29:05.785791 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.785770 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh\" (UID: \"0ccf23c2-7966-4d79-9865-d4ce4e485c6d\") " pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:29:05.793549 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.793525 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5z79\" (UniqueName: \"kubernetes.io/projected/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-kube-api-access-t5z79\") pod \"payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh\" (UID: \"0ccf23c2-7966-4d79-9865-d4ce4e485c6d\") " pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:29:05.821948 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:05.821915 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:29:06.055880 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:06.055857 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb"] Sep 08 07:29:06.057159 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:29:06.057133 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d0c9e54_71ee_4acd_99e4_cfca32ceb48d.slice/crio-d960eee9654d2f6bfb89332c3ab9f2328fcd70a8012a5a155d5911f5a86b50b2 WatchSource:0}: Error finding container d960eee9654d2f6bfb89332c3ab9f2328fcd70a8012a5a155d5911f5a86b50b2: Status 404 returned error can't find the container with id d960eee9654d2f6bfb89332c3ab9f2328fcd70a8012a5a155d5911f5a86b50b2 Sep 08 07:29:06.109247 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:06.109208 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh"] Sep 08 07:29:06.116346 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:29:06.116319 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ccf23c2_7966_4d79_9865_d4ce4e485c6d.slice/crio-1fe30bd58b48d8950f208a7c2f2b75123f3b1212e3a553170c0560e67297a8db WatchSource:0}: Error finding container 1fe30bd58b48d8950f208a7c2f2b75123f3b1212e3a553170c0560e67297a8db: Status 404 returned error can't find the container with id 1fe30bd58b48d8950f208a7c2f2b75123f3b1212e3a553170c0560e67297a8db Sep 08 07:29:06.862649 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:06.862612 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" event={"ID":"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d","Type":"ContainerStarted","Data":"0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4"} Sep 08 07:29:06.863135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:06.862676 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" event={"ID":"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d","Type":"ContainerStarted","Data":"d960eee9654d2f6bfb89332c3ab9f2328fcd70a8012a5a155d5911f5a86b50b2"} Sep 08 07:29:06.863135 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:06.862767 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:29:06.864261 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:06.864209 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" event={"ID":"0ccf23c2-7966-4d79-9865-d4ce4e485c6d","Type":"ContainerStarted","Data":"c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b"} Sep 08 07:29:06.864399 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:06.864285 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" event={"ID":"0ccf23c2-7966-4d79-9865-d4ce4e485c6d","Type":"ContainerStarted","Data":"1fe30bd58b48d8950f208a7c2f2b75123f3b1212e3a553170c0560e67297a8db"} Sep 08 07:29:06.864399 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:06.864320 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:29:06.880207 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:06.880153 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" podStartSLOduration=1.880138878 podStartE2EDuration="1.880138878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:29:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:29:06.878723966 +0000 UTC m=+1461.556687105" watchObservedRunningTime="2026-09-08 07:29:06.880138878 +0000 UTC m=+1461.558101994" Sep 08 07:29:06.897285 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:06.897232 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" podStartSLOduration=1.8972056720000001 podStartE2EDuration="1.897205672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:29:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:29:06.895396918 +0000 UTC m=+1461.573360061" watchObservedRunningTime="2026-09-08 07:29:06.897205672 +0000 UTC m=+1461.575168789" Sep 08 07:29:17.871398 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:17.871365 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:29:17.871981 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:17.871481 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:29:24.985012 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:24.984973 2571 generic.go:356] "Generic (PLEG): container finished" podID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerID="b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147" exitCode=0 Sep 08 07:29:24.985452 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:24.985043 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" event={"ID":"8b272ad8-0053-4aed-b6d3-4c160a1ca87a","Type":"ContainerDied","Data":"b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147"} Sep 08 07:29:24.985502 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:24.985482 2571 scope.go:117] "RemoveContainer" containerID="b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147" Sep 08 07:29:24.986581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:24.986548 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:29:24.986682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:24.986616 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:29:24.986682 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:24.986646 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:29:25.998199 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:25.998160 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" event={"ID":"8b272ad8-0053-4aed-b6d3-4c160a1ca87a","Type":"ContainerStarted","Data":"ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc"} Sep 08 07:29:26.479776 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:26.479735 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:26.480129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:26.480096 2571 patch_prober.go:28] interesting pod/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Sep 08 07:29:26.480239 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:26.480163 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Sep 08 07:29:27.479616 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:27.479568 2571 patch_prober.go:28] interesting pod/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Sep 08 07:29:27.480097 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:27.479662 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Sep 08 07:29:28.479853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:28.479807 2571 patch_prober.go:28] interesting pod/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Sep 08 07:29:28.480529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:28.480494 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Sep 08 07:29:29.479834 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:29.479795 2571 patch_prober.go:28] interesting pod/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Sep 08 07:29:29.480116 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:29.480084 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Sep 08 07:29:30.480259 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:30.480193 2571 patch_prober.go:28] interesting pod/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Sep 08 07:29:30.480762 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:30.480290 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Sep 08 07:29:31.506928 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:31.506887 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:31.507423 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:31.507400 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:31.508947 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:31.508922 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:29:46.016201 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:46.016164 2571 scope.go:117] "RemoveContainer" containerID="269c1bacf42eb5c1db61e27f0ac5cf9b2fbec3e33329d286dd0c5538402c6770" Sep 08 07:29:46.024990 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:46.024970 2571 scope.go:117] "RemoveContainer" containerID="cca46e5b0a2d08c1e98160a5ce874a420959120981a6a73e6bb41df304a97007" Sep 08 07:29:53.589781 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:53.589748 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h"] Sep 08 07:29:53.590284 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:53.589980 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" containerID="cri-o://0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538" gracePeriod=30 Sep 08 07:29:58.844398 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.844372 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:29:58.916482 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916409 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-socket\") pod \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " Sep 08 07:29:58.916482 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916467 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-data\") pod \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " Sep 08 07:29:58.916482 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916491 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-envoy\") pod \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " Sep 08 07:29:58.916775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916528 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8s482\" (UniqueName: \"kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-kube-api-access-8s482\") pod \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " Sep 08 07:29:58.916775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916552 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istiod-ca-cert\") pod \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " Sep 08 07:29:58.916775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916590 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-credential-socket\") pod \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " Sep 08 07:29:58.916775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916627 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-certs\") pod \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " Sep 08 07:29:58.916775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916662 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-token\") pod \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " Sep 08 07:29:58.916775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916684 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71266cd5-c7bd-40a7-870b-effcdd2f27aa" (UID: "71266cd5-c7bd-40a7-870b-effcdd2f27aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:29:58.916775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916703 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-podinfo\") pod \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\" (UID: \"71266cd5-c7bd-40a7-870b-effcdd2f27aa\") " Sep 08 07:29:58.917147 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916776 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "71266cd5-c7bd-40a7-870b-effcdd2f27aa" (UID: "71266cd5-c7bd-40a7-870b-effcdd2f27aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:29:58.917147 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916902 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71266cd5-c7bd-40a7-870b-effcdd2f27aa" (UID: "71266cd5-c7bd-40a7-870b-effcdd2f27aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:29:58.917147 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916918 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71266cd5-c7bd-40a7-870b-effcdd2f27aa" (UID: "71266cd5-c7bd-40a7-870b-effcdd2f27aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:29:58.917147 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.916972 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71266cd5-c7bd-40a7-870b-effcdd2f27aa" (UID: "71266cd5-c7bd-40a7-870b-effcdd2f27aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:29:58.917147 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.917146 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-data\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:29:58.917413 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.917162 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istiod-ca-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:29:58.917413 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.917171 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-credential-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:29:58.917413 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.917180 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:29:58.917413 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.917193 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-workload-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:29:58.918779 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.918758 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71266cd5-c7bd-40a7-870b-effcdd2f27aa" (UID: "71266cd5-c7bd-40a7-870b-effcdd2f27aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:29:58.918876 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.918858 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-kube-api-access-8s482" (OuterVolumeSpecName: "kube-api-access-8s482") pod "71266cd5-c7bd-40a7-870b-effcdd2f27aa" (UID: "71266cd5-c7bd-40a7-870b-effcdd2f27aa"). InnerVolumeSpecName "kube-api-access-8s482". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:29:58.919104 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.919088 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71266cd5-c7bd-40a7-870b-effcdd2f27aa" (UID: "71266cd5-c7bd-40a7-870b-effcdd2f27aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 07:29:58.919157 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:58.919104 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "71266cd5-c7bd-40a7-870b-effcdd2f27aa" (UID: "71266cd5-c7bd-40a7-870b-effcdd2f27aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:29:59.018534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.018490 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s482\" (UniqueName: \"kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-kube-api-access-8s482\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:29:59.018534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.018527 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-token\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:29:59.018534 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.018540 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-podinfo\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:29:59.018778 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.018551 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71266cd5-c7bd-40a7-870b-effcdd2f27aa-istio-envoy\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:29:59.199437 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.199400 2571 generic.go:356] "Generic (PLEG): container finished" podID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerID="0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538" exitCode=0 Sep 08 07:29:59.199604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.199449 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" Sep 08 07:29:59.199604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.199484 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" event={"ID":"71266cd5-c7bd-40a7-870b-effcdd2f27aa","Type":"ContainerDied","Data":"0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538"} Sep 08 07:29:59.199604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.199540 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h" event={"ID":"71266cd5-c7bd-40a7-870b-effcdd2f27aa","Type":"ContainerDied","Data":"94596e5443ba4888355fff5a0cadf7f53b6b5d7e93150aa5093a84f723c1e70e"} Sep 08 07:29:59.199604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.199565 2571 scope.go:117] "RemoveContainer" containerID="0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538" Sep 08 07:29:59.209372 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.209341 2571 scope.go:117] "RemoveContainer" containerID="bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe" Sep 08 07:29:59.217512 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.217494 2571 scope.go:117] "RemoveContainer" containerID="0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538" Sep 08 07:29:59.217779 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:29:59.217760 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538\": container with ID starting with 0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538 not found: ID does not exist" containerID="0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538" Sep 08 07:29:59.217827 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.217787 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538"} err="failed to get container status \"0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538\": rpc error: code = NotFound desc = could not find container \"0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538\": container with ID starting with 0ee57183d39ebbc6b56fe005f0705afdd1848cec2245a9685f9cb11f8eab3538 not found: ID does not exist" Sep 08 07:29:59.217827 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.217804 2571 scope.go:117] "RemoveContainer" containerID="bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe" Sep 08 07:29:59.218027 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:29:59.218009 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe\": container with ID starting with bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe not found: ID does not exist" containerID="bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe" Sep 08 07:29:59.218085 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.218036 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe"} err="failed to get container status \"bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe\": rpc error: code = NotFound desc = could not find container \"bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe\": container with ID starting with bfaba62ef586d3eb962ebd976529d30e24f462009ecdf71e5d3ab817062d27fe not found: ID does not exist" Sep 08 07:29:59.223133 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.223110 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h"] Sep 08 07:29:59.226509 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.226490 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-7e01ea-openshift-default-75684d4958-dvh6h"] Sep 08 07:29:59.809491 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:29:59.809461 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" path="/var/lib/kubelet/pods/71266cd5-c7bd-40a7-870b-effcdd2f27aa/volumes" Sep 08 07:30:00.137472 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.137395 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz"] Sep 08 07:30:00.137843 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.137829 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" Sep 08 07:30:00.137893 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.137846 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" Sep 08 07:30:00.137931 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.137920 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" Sep 08 07:30:00.140891 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.140859 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz" Sep 08 07:30:00.143524 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.143498 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-824nh\"" Sep 08 07:30:00.144054 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.144033 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb"] Sep 08 07:30:00.144511 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.144489 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" Sep 08 07:30:00.144511 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.144508 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" Sep 08 07:30:00.144671 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.144595 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="71266cd5-c7bd-40a7-870b-effcdd2f27aa" containerName="istio-proxy" Sep 08 07:30:00.147503 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.147485 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb" Sep 08 07:30:00.152755 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.152698 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz"] Sep 08 07:30:00.153612 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.153591 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz" Sep 08 07:30:00.163701 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.163680 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb" Sep 08 07:30:00.164194 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.164169 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb"] Sep 08 07:30:00.167165 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.167109 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f"] Sep 08 07:30:00.184502 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.184474 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f"] Sep 08 07:30:00.184739 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.184602 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f" Sep 08 07:30:00.221299 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.221274 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f" Sep 08 07:30:00.451228 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.451189 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz"] Sep 08 07:30:00.452195 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:30:00.452172 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53fa16e_e6e9_4ee5_8d13_8006218523fd.slice/crio-b78ce8c44d98eb8344b67ec8279aa6058594022c1761b3de1399a6022b9e11f9 WatchSource:0}: Error finding container b78ce8c44d98eb8344b67ec8279aa6058594022c1761b3de1399a6022b9e11f9: Status 404 returned error can't find the container with id b78ce8c44d98eb8344b67ec8279aa6058594022c1761b3de1399a6022b9e11f9 Sep 08 07:30:00.479254 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.479223 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb"] Sep 08 07:30:00.481376 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:30:00.481354 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72eb1972_dbd3_4fbc_a655_d44035eac274.slice/crio-999b2d8c36388948901f64659425b5d48a098cd76a630a8a96cd385a440e8951 WatchSource:0}: Error finding container 999b2d8c36388948901f64659425b5d48a098cd76a630a8a96cd385a440e8951: Status 404 returned error can't find the container with id 999b2d8c36388948901f64659425b5d48a098cd76a630a8a96cd385a440e8951 Sep 08 07:30:00.514281 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:00.514257 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f"] Sep 08 07:30:00.515112 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:30:00.515087 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f806e5_ecbb_433b_87fa_25b787949ac6.slice/crio-1c1266b23e5872e6e2e6005e2448c63468d27c304970d8adca80757bb21bd64b WatchSource:0}: Error finding container 1c1266b23e5872e6e2e6005e2448c63468d27c304970d8adca80757bb21bd64b: Status 404 returned error can't find the container with id 1c1266b23e5872e6e2e6005e2448c63468d27c304970d8adca80757bb21bd64b Sep 08 07:30:01.217848 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:01.217814 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb" event={"ID":"72eb1972-dbd3-4fbc-a655-d44035eac274","Type":"ContainerStarted","Data":"45910516a97b25fa41fdfa78eb6fc4dc6ce953b00f4dac764855cceece1a796d"} Sep 08 07:30:01.218326 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:01.217864 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb" event={"ID":"72eb1972-dbd3-4fbc-a655-d44035eac274","Type":"ContainerStarted","Data":"999b2d8c36388948901f64659425b5d48a098cd76a630a8a96cd385a440e8951"} Sep 08 07:30:01.219502 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:01.219476 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz" event={"ID":"c53fa16e-e6e9-4ee5-8d13-8006218523fd","Type":"ContainerStarted","Data":"227eb863493f032f8ec19eede434b6ac238fd7336d3668f23c3d64f1f0d91938"} Sep 08 07:30:01.219608 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:01.219520 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz" event={"ID":"c53fa16e-e6e9-4ee5-8d13-8006218523fd","Type":"ContainerStarted","Data":"b78ce8c44d98eb8344b67ec8279aa6058594022c1761b3de1399a6022b9e11f9"} Sep 08 07:30:01.221048 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:01.221026 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f" event={"ID":"e9f806e5-ecbb-433b-87fa-25b787949ac6","Type":"ContainerStarted","Data":"20e6cf3023bbc1b2bd73d6d4333299b3398fd485ea3ab298ad85471fdcb0b33e"} Sep 08 07:30:01.221137 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:01.221064 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f" event={"ID":"e9f806e5-ecbb-433b-87fa-25b787949ac6","Type":"ContainerStarted","Data":"1c1266b23e5872e6e2e6005e2448c63468d27c304970d8adca80757bb21bd64b"} Sep 08 07:30:01.234490 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:01.234446 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb" podStartSLOduration=1.234432021 podStartE2EDuration="1.234432021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:30:01.232786398 +0000 UTC m=+1515.910749515" watchObservedRunningTime="2026-09-08 07:30:01.234432021 +0000 UTC m=+1515.912395136" Sep 08 07:30:01.248729 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:01.248679 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f" podStartSLOduration=1.248669389 podStartE2EDuration="1.248669389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:30:01.246430004 +0000 UTC m=+1515.924393142" watchObservedRunningTime="2026-09-08 07:30:01.248669389 +0000 UTC m=+1515.926632504" Sep 08 07:30:01.261088 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:01.261043 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz" podStartSLOduration=1.261028385 podStartE2EDuration="1.261028385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:30:01.259762727 +0000 UTC m=+1515.937725864" watchObservedRunningTime="2026-09-08 07:30:01.261028385 +0000 UTC m=+1515.938991500" Sep 08 07:30:02.228385 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:02.228350 2571 generic.go:356] "Generic (PLEG): container finished" podID="72eb1972-dbd3-4fbc-a655-d44035eac274" containerID="45910516a97b25fa41fdfa78eb6fc4dc6ce953b00f4dac764855cceece1a796d" exitCode=0 Sep 08 07:30:02.228839 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:02.228443 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb" event={"ID":"72eb1972-dbd3-4fbc-a655-d44035eac274","Type":"ContainerDied","Data":"45910516a97b25fa41fdfa78eb6fc4dc6ce953b00f4dac764855cceece1a796d"} Sep 08 07:30:02.230000 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:02.229979 2571 generic.go:356] "Generic (PLEG): container finished" podID="c53fa16e-e6e9-4ee5-8d13-8006218523fd" containerID="227eb863493f032f8ec19eede434b6ac238fd7336d3668f23c3d64f1f0d91938" exitCode=0 Sep 08 07:30:02.230108 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:02.230005 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz" event={"ID":"c53fa16e-e6e9-4ee5-8d13-8006218523fd","Type":"ContainerDied","Data":"227eb863493f032f8ec19eede434b6ac238fd7336d3668f23c3d64f1f0d91938"} Sep 08 07:30:02.231466 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:02.231444 2571 generic.go:356] "Generic (PLEG): container finished" podID="e9f806e5-ecbb-433b-87fa-25b787949ac6" containerID="20e6cf3023bbc1b2bd73d6d4333299b3398fd485ea3ab298ad85471fdcb0b33e" exitCode=0 Sep 08 07:30:02.231553 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:02.231478 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f" event={"ID":"e9f806e5-ecbb-433b-87fa-25b787949ac6","Type":"ContainerDied","Data":"20e6cf3023bbc1b2bd73d6d4333299b3398fd485ea3ab298ad85471fdcb0b33e"} Sep 08 07:30:03.440095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:03.440071 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz" Sep 08 07:30:03.443603 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:03.443586 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb" Sep 08 07:30:03.446676 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:03.446661 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f" Sep 08 07:30:04.243922 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:04.243895 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb" Sep 08 07:30:04.243922 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:04.243904 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb" event={"ID":"72eb1972-dbd3-4fbc-a655-d44035eac274","Type":"ContainerDied","Data":"999b2d8c36388948901f64659425b5d48a098cd76a630a8a96cd385a440e8951"} Sep 08 07:30:04.244158 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:04.243954 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="999b2d8c36388948901f64659425b5d48a098cd76a630a8a96cd385a440e8951" Sep 08 07:30:04.245431 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:04.245410 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz" Sep 08 07:30:04.245561 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:04.245429 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz" event={"ID":"c53fa16e-e6e9-4ee5-8d13-8006218523fd","Type":"ContainerDied","Data":"b78ce8c44d98eb8344b67ec8279aa6058594022c1761b3de1399a6022b9e11f9"} Sep 08 07:30:04.245561 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:04.245468 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b78ce8c44d98eb8344b67ec8279aa6058594022c1761b3de1399a6022b9e11f9" Sep 08 07:30:04.250333 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:04.250316 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f" Sep 08 07:30:04.250333 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:04.250331 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814210-5nx9f" event={"ID":"e9f806e5-ecbb-433b-87fa-25b787949ac6","Type":"ContainerDied","Data":"1c1266b23e5872e6e2e6005e2448c63468d27c304970d8adca80757bb21bd64b"} Sep 08 07:30:04.250486 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:04.250363 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c1266b23e5872e6e2e6005e2448c63468d27c304970d8adca80757bb21bd64b" Sep 08 07:30:09.487133 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.487044 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz"] Sep 08 07:30:09.489954 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.489927 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f0e0d8-29814210-mkztz"] Sep 08 07:30:09.512262 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.512228 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh"] Sep 08 07:30:09.512519 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.512472 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" podUID="0ccf23c2-7966-4d79-9865-d4ce4e485c6d" containerName="payload-processing" containerID="cri-o://c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b" gracePeriod=30 Sep 08 07:30:09.519983 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.519958 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb"] Sep 08 07:30:09.520470 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.520382 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" podUID="8d0c9e54-71ee-4acd-99e4-cfca32ceb48d" containerName="payload-pre-processing" containerID="cri-o://0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4" gracePeriod=30 Sep 08 07:30:09.766587 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.766562 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:30:09.801205 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.801184 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:30:09.810086 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.810059 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c53fa16e-e6e9-4ee5-8d13-8006218523fd" path="/var/lib/kubelet/pods/c53fa16e-e6e9-4ee5-8d13-8006218523fd/volumes" Sep 08 07:30:09.833970 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.833003 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5z79\" (UniqueName: \"kubernetes.io/projected/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-kube-api-access-t5z79\") pod \"0ccf23c2-7966-4d79-9865-d4ce4e485c6d\" (UID: \"0ccf23c2-7966-4d79-9865-d4ce4e485c6d\") " Sep 08 07:30:09.833970 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.833208 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-plugins-config-volume\") pod \"0ccf23c2-7966-4d79-9865-d4ce4e485c6d\" (UID: \"0ccf23c2-7966-4d79-9865-d4ce4e485c6d\") " Sep 08 07:30:09.833970 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.833912 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0ccf23c2-7966-4d79-9865-d4ce4e485c6d" (UID: "0ccf23c2-7966-4d79-9865-d4ce4e485c6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:30:09.836878 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.836832 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-kube-api-access-t5z79" (OuterVolumeSpecName: "kube-api-access-t5z79") pod "0ccf23c2-7966-4d79-9865-d4ce4e485c6d" (UID: "0ccf23c2-7966-4d79-9865-d4ce4e485c6d"). InnerVolumeSpecName "kube-api-access-t5z79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:30:09.934522 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.934484 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-plugins-config-volume\") pod \"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d\" (UID: \"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d\") " Sep 08 07:30:09.934694 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.934576 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg9pr\" (UniqueName: \"kubernetes.io/projected/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-kube-api-access-mg9pr\") pod \"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d\" (UID: \"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d\") " Sep 08 07:30:09.934799 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.934787 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:09.934834 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.934801 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5z79\" (UniqueName: \"kubernetes.io/projected/0ccf23c2-7966-4d79-9865-d4ce4e485c6d-kube-api-access-t5z79\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:09.934873 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.934847 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d0c9e54-71ee-4acd-99e4-cfca32ceb48d" (UID: "8d0c9e54-71ee-4acd-99e4-cfca32ceb48d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:30:09.936577 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:09.936558 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-kube-api-access-mg9pr" (OuterVolumeSpecName: "kube-api-access-mg9pr") pod "8d0c9e54-71ee-4acd-99e4-cfca32ceb48d" (UID: "8d0c9e54-71ee-4acd-99e4-cfca32ceb48d"). InnerVolumeSpecName "kube-api-access-mg9pr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:30:10.036376 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.036287 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:10.036376 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.036316 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg9pr\" (UniqueName: \"kubernetes.io/projected/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d-kube-api-access-mg9pr\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:10.285226 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.285180 2571 generic.go:356] "Generic (PLEG): container finished" podID="0ccf23c2-7966-4d79-9865-d4ce4e485c6d" containerID="c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b" exitCode=0 Sep 08 07:30:10.285388 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.285245 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" Sep 08 07:30:10.285388 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.285265 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" event={"ID":"0ccf23c2-7966-4d79-9865-d4ce4e485c6d","Type":"ContainerDied","Data":"c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b"} Sep 08 07:30:10.285388 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.285318 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh" event={"ID":"0ccf23c2-7966-4d79-9865-d4ce4e485c6d","Type":"ContainerDied","Data":"1fe30bd58b48d8950f208a7c2f2b75123f3b1212e3a553170c0560e67297a8db"} Sep 08 07:30:10.285388 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.285343 2571 scope.go:117] "RemoveContainer" containerID="c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b" Sep 08 07:30:10.286965 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.286899 2571 generic.go:356] "Generic (PLEG): container finished" podID="8d0c9e54-71ee-4acd-99e4-cfca32ceb48d" containerID="0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4" exitCode=0 Sep 08 07:30:10.286965 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.286933 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" event={"ID":"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d","Type":"ContainerDied","Data":"0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4"} Sep 08 07:30:10.286965 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.286941 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" Sep 08 07:30:10.287158 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.286973 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb" event={"ID":"8d0c9e54-71ee-4acd-99e4-cfca32ceb48d","Type":"ContainerDied","Data":"d960eee9654d2f6bfb89332c3ab9f2328fcd70a8012a5a155d5911f5a86b50b2"} Sep 08 07:30:10.295331 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.295207 2571 scope.go:117] "RemoveContainer" containerID="c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b" Sep 08 07:30:10.295657 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:30:10.295639 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b\": container with ID starting with c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b not found: ID does not exist" containerID="c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b" Sep 08 07:30:10.295719 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.295666 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b"} err="failed to get container status \"c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b\": rpc error: code = NotFound desc = could not find container \"c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b\": container with ID starting with c1578388267225e0a9aec4db4fbb1bd91fda9e8716bd8ca27da3b491cb64f53b not found: ID does not exist" Sep 08 07:30:10.295719 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.295686 2571 scope.go:117] "RemoveContainer" containerID="0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4" Sep 08 07:30:10.303940 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.303925 2571 scope.go:117] "RemoveContainer" containerID="0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4" Sep 08 07:30:10.304191 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:30:10.304173 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4\": container with ID starting with 0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4 not found: ID does not exist" containerID="0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4" Sep 08 07:30:10.304269 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.304198 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4"} err="failed to get container status \"0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4\": rpc error: code = NotFound desc = could not find container \"0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4\": container with ID starting with 0f88c15c99bba29f142f7d8eab43957e5138092a0836bc7cf5e7ccf040f5dee4 not found: ID does not exist" Sep 08 07:30:10.325263 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.325239 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb"] Sep 08 07:30:10.327702 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.327682 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f0e0d8-9b7b8d57d-4pxnb"] Sep 08 07:30:10.338161 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.338136 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh"] Sep 08 07:30:10.343153 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:10.343131 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f0e0d8-7bfbb54d8c-7lrdh"] Sep 08 07:30:11.809266 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:11.809203 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ccf23c2-7966-4d79-9865-d4ce4e485c6d" path="/var/lib/kubelet/pods/0ccf23c2-7966-4d79-9865-d4ce4e485c6d/volumes" Sep 08 07:30:11.809815 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:11.809794 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d0c9e54-71ee-4acd-99e4-cfca32ceb48d" path="/var/lib/kubelet/pods/8d0c9e54-71ee-4acd-99e4-cfca32ceb48d/volumes" Sep 08 07:30:16.607505 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:16.607472 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc"] Sep 08 07:30:16.607919 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:16.607692 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" containerID="cri-o://ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc" gracePeriod=30 Sep 08 07:30:21.864394 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.864364 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:30:21.952496 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952463 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-credential-socket\") pod \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " Sep 08 07:30:21.952671 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952504 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istiod-ca-cert\") pod \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " Sep 08 07:30:21.952671 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952531 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr9tl\" (UniqueName: \"kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-kube-api-access-pr9tl\") pod \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " Sep 08 07:30:21.952671 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952563 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-certs\") pod \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " Sep 08 07:30:21.952671 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952603 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-data\") pod \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " Sep 08 07:30:21.952671 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952640 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-podinfo\") pod \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " Sep 08 07:30:21.952943 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952692 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-token\") pod \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " Sep 08 07:30:21.952943 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952741 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-socket\") pod \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " Sep 08 07:30:21.952943 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952752 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b272ad8-0053-4aed-b6d3-4c160a1ca87a" (UID: "8b272ad8-0053-4aed-b6d3-4c160a1ca87a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:30:21.952943 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952810 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-envoy\") pod \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\" (UID: \"8b272ad8-0053-4aed-b6d3-4c160a1ca87a\") " Sep 08 07:30:21.952943 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952865 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b272ad8-0053-4aed-b6d3-4c160a1ca87a" (UID: "8b272ad8-0053-4aed-b6d3-4c160a1ca87a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:30:21.952943 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952900 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b272ad8-0053-4aed-b6d3-4c160a1ca87a" (UID: "8b272ad8-0053-4aed-b6d3-4c160a1ca87a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:30:21.952943 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.952913 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b272ad8-0053-4aed-b6d3-4c160a1ca87a" (UID: "8b272ad8-0053-4aed-b6d3-4c160a1ca87a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:30:21.953355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.953020 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b272ad8-0053-4aed-b6d3-4c160a1ca87a" (UID: "8b272ad8-0053-4aed-b6d3-4c160a1ca87a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:30:21.953355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.953203 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-credential-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:21.953355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.953251 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istiod-ca-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:21.953355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.953267 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:21.953355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.953279 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-data\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:21.953355 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.953293 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-workload-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:21.954895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.954867 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b272ad8-0053-4aed-b6d3-4c160a1ca87a" (UID: "8b272ad8-0053-4aed-b6d3-4c160a1ca87a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:30:21.955069 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.955045 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b272ad8-0053-4aed-b6d3-4c160a1ca87a" (UID: "8b272ad8-0053-4aed-b6d3-4c160a1ca87a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:30:21.955146 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.955045 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-kube-api-access-pr9tl" (OuterVolumeSpecName: "kube-api-access-pr9tl") pod "8b272ad8-0053-4aed-b6d3-4c160a1ca87a" (UID: "8b272ad8-0053-4aed-b6d3-4c160a1ca87a"). InnerVolumeSpecName "kube-api-access-pr9tl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:30:21.955146 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:21.955050 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b272ad8-0053-4aed-b6d3-4c160a1ca87a" (UID: "8b272ad8-0053-4aed-b6d3-4c160a1ca87a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 07:30:22.053947 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.053909 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-podinfo\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:22.053947 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.053939 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-token\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:22.053947 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.053949 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-istio-envoy\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:22.054178 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.053957 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr9tl\" (UniqueName: \"kubernetes.io/projected/8b272ad8-0053-4aed-b6d3-4c160a1ca87a-kube-api-access-pr9tl\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:22.358761 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.358673 2571 generic.go:356] "Generic (PLEG): container finished" podID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerID="ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc" exitCode=0 Sep 08 07:30:22.358761 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.358725 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" Sep 08 07:30:22.358960 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.358761 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" event={"ID":"8b272ad8-0053-4aed-b6d3-4c160a1ca87a","Type":"ContainerDied","Data":"ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc"} Sep 08 07:30:22.358960 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.358819 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc" event={"ID":"8b272ad8-0053-4aed-b6d3-4c160a1ca87a","Type":"ContainerDied","Data":"5b68e285a40ba86ca92b6de6db2b559056a8feeea7bebcf02e326ee9fbc778af"} Sep 08 07:30:22.358960 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.358842 2571 scope.go:117] "RemoveContainer" containerID="ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc" Sep 08 07:30:22.368895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.368874 2571 scope.go:117] "RemoveContainer" containerID="b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147" Sep 08 07:30:22.377317 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.377178 2571 scope.go:117] "RemoveContainer" containerID="ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc" Sep 08 07:30:22.377584 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:30:22.377557 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc\": container with ID starting with ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc not found: ID does not exist" containerID="ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc" Sep 08 07:30:22.377677 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.377596 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc"} err="failed to get container status \"ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc\": rpc error: code = NotFound desc = could not find container \"ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc\": container with ID starting with ded3844150638a8e68bc9b686397b97682a12c50beb292214ae17b875fcbdabc not found: ID does not exist" Sep 08 07:30:22.377677 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.377621 2571 scope.go:117] "RemoveContainer" containerID="b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147" Sep 08 07:30:22.378137 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:30:22.378008 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147\": container with ID starting with b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147 not found: ID does not exist" containerID="b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147" Sep 08 07:30:22.378137 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.378065 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147"} err="failed to get container status \"b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147\": rpc error: code = NotFound desc = could not find container \"b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147\": container with ID starting with b68a509d11431ca4c393ba0fbec9db8e504f0de8c6e33beec06d63213b6de147 not found: ID does not exist" Sep 08 07:30:22.380353 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.380335 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc"] Sep 08 07:30:22.384060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:22.384040 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f0e0d8-openshift-default-7d7567fc7b-rspgc"] Sep 08 07:30:23.815164 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:23.815119 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" path="/var/lib/kubelet/pods/8b272ad8-0053-4aed-b6d3-4c160a1ca87a/volumes" Sep 08 07:30:27.518742 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:27.518708 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l"] Sep 08 07:30:27.519456 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:27.519422 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" podUID="dc378680-0ed1-4273-82f1-838ee35c5a9d" containerName="payload-processing" containerID="cri-o://eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c" gracePeriod=30 Sep 08 07:30:27.771711 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:27.771649 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:30:27.911536 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:27.911497 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qpvq\" (UniqueName: \"kubernetes.io/projected/dc378680-0ed1-4273-82f1-838ee35c5a9d-kube-api-access-7qpvq\") pod \"dc378680-0ed1-4273-82f1-838ee35c5a9d\" (UID: \"dc378680-0ed1-4273-82f1-838ee35c5a9d\") " Sep 08 07:30:27.911749 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:27.911563 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc378680-0ed1-4273-82f1-838ee35c5a9d-plugins-config-volume\") pod \"dc378680-0ed1-4273-82f1-838ee35c5a9d\" (UID: \"dc378680-0ed1-4273-82f1-838ee35c5a9d\") " Sep 08 07:30:27.911950 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:27.911921 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc378680-0ed1-4273-82f1-838ee35c5a9d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc378680-0ed1-4273-82f1-838ee35c5a9d" (UID: "dc378680-0ed1-4273-82f1-838ee35c5a9d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:30:27.913634 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:27.913605 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc378680-0ed1-4273-82f1-838ee35c5a9d-kube-api-access-7qpvq" (OuterVolumeSpecName: "kube-api-access-7qpvq") pod "dc378680-0ed1-4273-82f1-838ee35c5a9d" (UID: "dc378680-0ed1-4273-82f1-838ee35c5a9d"). InnerVolumeSpecName "kube-api-access-7qpvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:30:28.012422 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.012386 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qpvq\" (UniqueName: \"kubernetes.io/projected/dc378680-0ed1-4273-82f1-838ee35c5a9d-kube-api-access-7qpvq\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:28.012422 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.012419 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc378680-0ed1-4273-82f1-838ee35c5a9d-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:28.397336 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.397304 2571 generic.go:356] "Generic (PLEG): container finished" podID="dc378680-0ed1-4273-82f1-838ee35c5a9d" containerID="eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c" exitCode=0 Sep 08 07:30:28.397556 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.397354 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" Sep 08 07:30:28.397556 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.397400 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" event={"ID":"dc378680-0ed1-4273-82f1-838ee35c5a9d","Type":"ContainerDied","Data":"eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c"} Sep 08 07:30:28.397556 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.397455 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l" event={"ID":"dc378680-0ed1-4273-82f1-838ee35c5a9d","Type":"ContainerDied","Data":"577c1746b93b766016498a5ab3ab24f0d3faef8670a1dcf0aa573008cc18bd94"} Sep 08 07:30:28.397556 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.397484 2571 scope.go:117] "RemoveContainer" containerID="eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c" Sep 08 07:30:28.406420 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.406401 2571 scope.go:117] "RemoveContainer" containerID="eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c" Sep 08 07:30:28.406688 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:30:28.406671 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c\": container with ID starting with eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c not found: ID does not exist" containerID="eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c" Sep 08 07:30:28.406741 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.406696 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c"} err="failed to get container status \"eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c\": rpc error: code = NotFound desc = could not find container \"eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c\": container with ID starting with eea31de36da474bfc6c65cd018ac62873fd24c2624ce3e9c8ae0b166887acd1c not found: ID does not exist" Sep 08 07:30:28.419175 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.419151 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l"] Sep 08 07:30:28.422830 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:28.422809 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-37abc6-6ff58f8d45-jdj7l"] Sep 08 07:30:29.809869 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:29.809835 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc378680-0ed1-4273-82f1-838ee35c5a9d" path="/var/lib/kubelet/pods/dc378680-0ed1-4273-82f1-838ee35c5a9d/volumes" Sep 08 07:30:53.066533 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:53.066493 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v"] Sep 08 07:30:53.067038 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:53.066707 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" containerID="cri-o://390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d" gracePeriod=30 Sep 08 07:30:58.310099 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.310073 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:30:58.403815 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.403737 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-podinfo\") pod \"80809f81-3613-4de4-939a-3e32f6a1c87a\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " Sep 08 07:30:58.403815 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.403775 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-socket\") pod \"80809f81-3613-4de4-939a-3e32f6a1c87a\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " Sep 08 07:30:58.404121 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.403828 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5v2p\" (UniqueName: \"kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-kube-api-access-r5v2p\") pod \"80809f81-3613-4de4-939a-3e32f6a1c87a\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " Sep 08 07:30:58.404121 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.403861 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-envoy\") pod \"80809f81-3613-4de4-939a-3e32f6a1c87a\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " Sep 08 07:30:58.404121 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.403878 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-data\") pod \"80809f81-3613-4de4-939a-3e32f6a1c87a\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " Sep 08 07:30:58.404121 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.403905 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-token\") pod \"80809f81-3613-4de4-939a-3e32f6a1c87a\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " Sep 08 07:30:58.404121 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.403930 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80809f81-3613-4de4-939a-3e32f6a1c87a-istiod-ca-cert\") pod \"80809f81-3613-4de4-939a-3e32f6a1c87a\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " Sep 08 07:30:58.404121 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.403959 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-certs\") pod \"80809f81-3613-4de4-939a-3e32f6a1c87a\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " Sep 08 07:30:58.404509 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404164 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "80809f81-3613-4de4-939a-3e32f6a1c87a" (UID: "80809f81-3613-4de4-939a-3e32f6a1c87a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:30:58.404509 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404238 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "80809f81-3613-4de4-939a-3e32f6a1c87a" (UID: "80809f81-3613-4de4-939a-3e32f6a1c87a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:30:58.404509 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404266 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-credential-socket\") pod \"80809f81-3613-4de4-939a-3e32f6a1c87a\" (UID: \"80809f81-3613-4de4-939a-3e32f6a1c87a\") " Sep 08 07:30:58.404509 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404352 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80809f81-3613-4de4-939a-3e32f6a1c87a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "80809f81-3613-4de4-939a-3e32f6a1c87a" (UID: "80809f81-3613-4de4-939a-3e32f6a1c87a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:30:58.404509 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404499 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "80809f81-3613-4de4-939a-3e32f6a1c87a" (UID: "80809f81-3613-4de4-939a-3e32f6a1c87a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:30:58.404767 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404545 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-data" (OuterVolumeSpecName: "istio-data") pod "80809f81-3613-4de4-939a-3e32f6a1c87a" (UID: "80809f81-3613-4de4-939a-3e32f6a1c87a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:30:58.404767 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404668 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:58.404767 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404685 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-data\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:58.404767 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404700 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80809f81-3613-4de4-939a-3e32f6a1c87a-istiod-ca-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:58.404767 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404712 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-workload-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:58.404767 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.404725 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-credential-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:58.406513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.406472 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-token" (OuterVolumeSpecName: "istio-token") pod "80809f81-3613-4de4-939a-3e32f6a1c87a" (UID: "80809f81-3613-4de4-939a-3e32f6a1c87a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:30:58.406513 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.406490 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-kube-api-access-r5v2p" (OuterVolumeSpecName: "kube-api-access-r5v2p") pod "80809f81-3613-4de4-939a-3e32f6a1c87a" (UID: "80809f81-3613-4de4-939a-3e32f6a1c87a"). InnerVolumeSpecName "kube-api-access-r5v2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:30:58.406661 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.406527 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "80809f81-3613-4de4-939a-3e32f6a1c87a" (UID: "80809f81-3613-4de4-939a-3e32f6a1c87a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:30:58.406661 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.406546 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "80809f81-3613-4de4-939a-3e32f6a1c87a" (UID: "80809f81-3613-4de4-939a-3e32f6a1c87a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 07:30:58.505541 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.505509 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-podinfo\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:58.505541 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.505535 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5v2p\" (UniqueName: \"kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-kube-api-access-r5v2p\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:58.505541 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.505545 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-envoy\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:58.505755 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.505553 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80809f81-3613-4de4-939a-3e32f6a1c87a-istio-token\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:30:58.573806 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.573771 2571 generic.go:356] "Generic (PLEG): container finished" podID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerID="390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d" exitCode=0 Sep 08 07:30:58.573964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.573815 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" Sep 08 07:30:58.573964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.573836 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" event={"ID":"80809f81-3613-4de4-939a-3e32f6a1c87a","Type":"ContainerDied","Data":"390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d"} Sep 08 07:30:58.573964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.573883 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v" event={"ID":"80809f81-3613-4de4-939a-3e32f6a1c87a","Type":"ContainerDied","Data":"d7bc5f169609e0bee99605c2dd408c1d6ec703178253ae26e86dff4b70f3c7cc"} Sep 08 07:30:58.573964 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.573908 2571 scope.go:117] "RemoveContainer" containerID="390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d" Sep 08 07:30:58.583886 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.583870 2571 scope.go:117] "RemoveContainer" containerID="e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607" Sep 08 07:30:58.592062 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.592043 2571 scope.go:117] "RemoveContainer" containerID="390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d" Sep 08 07:30:58.592347 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:30:58.592319 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d\": container with ID starting with 390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d not found: ID does not exist" containerID="390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d" Sep 08 07:30:58.592420 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.592354 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d"} err="failed to get container status \"390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d\": rpc error: code = NotFound desc = could not find container \"390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d\": container with ID starting with 390c24fa230849896e4f6d2be95130113a7a3d85811cc80e1e270a1050bb010d not found: ID does not exist" Sep 08 07:30:58.592420 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.592370 2571 scope.go:117] "RemoveContainer" containerID="e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607" Sep 08 07:30:58.592591 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:30:58.592576 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607\": container with ID starting with e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607 not found: ID does not exist" containerID="e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607" Sep 08 07:30:58.592635 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.592594 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607"} err="failed to get container status \"e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607\": rpc error: code = NotFound desc = could not find container \"e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607\": container with ID starting with e50c670a095aa636f4b2b92f43942b975fd3b36966a3a818571f6bfb99832607 not found: ID does not exist" Sep 08 07:30:58.597442 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.597419 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v"] Sep 08 07:30:58.603941 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:58.603916 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ca807-openshift-default-697475d58b-rtl4v"] Sep 08 07:30:59.809196 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:30:59.809162 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" path="/var/lib/kubelet/pods/80809f81-3613-4de4-939a-3e32f6a1c87a/volumes" Sep 08 07:31:29.852303 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852268 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx"] Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852652 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72eb1972-dbd3-4fbc-a655-d44035eac274" containerName="cleanup" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852665 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="72eb1972-dbd3-4fbc-a655-d44035eac274" containerName="cleanup" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852673 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c53fa16e-e6e9-4ee5-8d13-8006218523fd" containerName="cleanup" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852678 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c53fa16e-e6e9-4ee5-8d13-8006218523fd" containerName="cleanup" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852690 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d0c9e54-71ee-4acd-99e4-cfca32ceb48d" containerName="payload-pre-processing" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852696 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d0c9e54-71ee-4acd-99e4-cfca32ceb48d" containerName="payload-pre-processing" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852704 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852709 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852716 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9f806e5-ecbb-433b-87fa-25b787949ac6" containerName="cleanup" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852721 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9f806e5-ecbb-433b-87fa-25b787949ac6" containerName="cleanup" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852729 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc378680-0ed1-4273-82f1-838ee35c5a9d" containerName="payload-processing" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852734 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc378680-0ed1-4273-82f1-838ee35c5a9d" containerName="payload-processing" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852743 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852748 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852754 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ccf23c2-7966-4d79-9865-d4ce4e485c6d" containerName="payload-processing" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852759 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ccf23c2-7966-4d79-9865-d4ce4e485c6d" containerName="payload-processing" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852767 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" Sep 08 07:31:29.852780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852772 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" Sep 08 07:31:29.853352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852831 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" Sep 08 07:31:29.853352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852840 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ccf23c2-7966-4d79-9865-d4ce4e485c6d" containerName="payload-processing" Sep 08 07:31:29.853352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852845 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9f806e5-ecbb-433b-87fa-25b787949ac6" containerName="cleanup" Sep 08 07:31:29.853352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852852 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" Sep 08 07:31:29.853352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852858 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" Sep 08 07:31:29.853352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852863 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d0c9e54-71ee-4acd-99e4-cfca32ceb48d" containerName="payload-pre-processing" Sep 08 07:31:29.853352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852870 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc378680-0ed1-4273-82f1-838ee35c5a9d" containerName="payload-processing" Sep 08 07:31:29.853352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852874 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="72eb1972-dbd3-4fbc-a655-d44035eac274" containerName="cleanup" Sep 08 07:31:29.853352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.852879 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c53fa16e-e6e9-4ee5-8d13-8006218523fd" containerName="cleanup" Sep 08 07:31:29.856132 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.856109 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:31:29.858568 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.858545 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-4d980e-dockercfg-wldqd\"" Sep 08 07:31:29.858700 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.858646 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-4d980e\"" Sep 08 07:31:29.864030 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.863938 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx"] Sep 08 07:31:29.989811 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.989773 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2wj8\" (UniqueName: \"kubernetes.io/projected/b86bc09a-26b7-462f-823c-260ad2f0d54e-kube-api-access-b2wj8\") pod \"payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx\" (UID: \"b86bc09a-26b7-462f-823c-260ad2f0d54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:31:29.989974 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:29.989918 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b86bc09a-26b7-462f-823c-260ad2f0d54e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx\" (UID: \"b86bc09a-26b7-462f-823c-260ad2f0d54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:31:30.090914 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:30.090877 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2wj8\" (UniqueName: \"kubernetes.io/projected/b86bc09a-26b7-462f-823c-260ad2f0d54e-kube-api-access-b2wj8\") pod \"payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx\" (UID: \"b86bc09a-26b7-462f-823c-260ad2f0d54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:31:30.091082 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:30.090977 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b86bc09a-26b7-462f-823c-260ad2f0d54e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx\" (UID: \"b86bc09a-26b7-462f-823c-260ad2f0d54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:31:30.091658 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:30.091639 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b86bc09a-26b7-462f-823c-260ad2f0d54e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx\" (UID: \"b86bc09a-26b7-462f-823c-260ad2f0d54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:31:30.100557 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:30.100530 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2wj8\" (UniqueName: \"kubernetes.io/projected/b86bc09a-26b7-462f-823c-260ad2f0d54e-kube-api-access-b2wj8\") pod \"payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx\" (UID: \"b86bc09a-26b7-462f-823c-260ad2f0d54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:31:30.167845 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:30.167758 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:31:30.439968 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:30.439943 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx"] Sep 08 07:31:30.441788 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:31:30.441763 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb86bc09a_26b7_462f_823c_260ad2f0d54e.slice/crio-28cf4cb4be9bf2511da7613b6506b1a130640ffdd517078bcce067c84d1ee241 WatchSource:0}: Error finding container 28cf4cb4be9bf2511da7613b6506b1a130640ffdd517078bcce067c84d1ee241: Status 404 returned error can't find the container with id 28cf4cb4be9bf2511da7613b6506b1a130640ffdd517078bcce067c84d1ee241 Sep 08 07:31:30.753779 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:30.753730 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" event={"ID":"b86bc09a-26b7-462f-823c-260ad2f0d54e","Type":"ContainerStarted","Data":"9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8"} Sep 08 07:31:30.753980 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:30.753790 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" event={"ID":"b86bc09a-26b7-462f-823c-260ad2f0d54e","Type":"ContainerStarted","Data":"28cf4cb4be9bf2511da7613b6506b1a130640ffdd517078bcce067c84d1ee241"} Sep 08 07:31:30.753980 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:30.753907 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:31:30.770193 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:30.770151 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" podStartSLOduration=1.770138809 podStartE2EDuration="1.770138809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:31:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:31:30.76885349 +0000 UTC m=+1605.446816635" watchObservedRunningTime="2026-09-08 07:31:30.770138809 +0000 UTC m=+1605.448101926" Sep 08 07:31:41.761810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:41.761775 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:31:53.963021 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:53.962981 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz"] Sep 08 07:31:53.963527 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:53.963412 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" Sep 08 07:31:53.963527 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:53.963424 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="80809f81-3613-4de4-939a-3e32f6a1c87a" containerName="istio-proxy" Sep 08 07:31:53.963527 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:53.963481 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b272ad8-0053-4aed-b6d3-4c160a1ca87a" containerName="istio-proxy" Sep 08 07:31:53.966585 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:53.966567 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:53.969812 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:53.969781 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-79da86-openshift-default-dockercfg-h9lz6\"" Sep 08 07:31:53.984259 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:53.984232 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz"] Sep 08 07:31:54.005826 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.005787 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-istio-token\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.005996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.005859 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-envoy\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.005996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.005890 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-credential-socket\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.005996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.005916 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6142a33b-fd87-4fca-8464-4f7604658f61-istiod-ca-cert\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.005996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.005952 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6142a33b-fd87-4fca-8464-4f7604658f61-istio-podinfo\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.006156 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.006031 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-socket\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.006156 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.006061 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmhst\" (UniqueName: \"kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-kube-api-access-nmhst\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.006156 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.006094 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-data\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.006156 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.006118 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-certs\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.106951 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.106913 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-socket\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107157 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.106960 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmhst\" (UniqueName: \"kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-kube-api-access-nmhst\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107157 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.106984 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-data\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107157 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107001 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-certs\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107157 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107025 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-istio-token\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107157 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107052 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-envoy\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107479 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107198 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-credential-socket\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107479 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107254 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6142a33b-fd87-4fca-8464-4f7604658f61-istiod-ca-cert\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107479 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107284 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6142a33b-fd87-4fca-8464-4f7604658f61-istio-podinfo\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107479 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107370 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-socket\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107479 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107445 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-data\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107742 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107551 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-certs\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107742 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107580 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-credential-socket\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.107900 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.107878 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6142a33b-fd87-4fca-8464-4f7604658f61-istiod-ca-cert\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.109190 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.109171 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-envoy\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.109779 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.109752 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6142a33b-fd87-4fca-8464-4f7604658f61-istio-podinfo\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.117100 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.117072 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-istio-token\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.117956 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.117933 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmhst\" (UniqueName: \"kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-kube-api-access-nmhst\") pod \"e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.287277 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.287186 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:54.567464 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.567436 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz"] Sep 08 07:31:54.568524 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:31:54.568496 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6142a33b_fd87_4fca_8464_4f7604658f61.slice/crio-5bfe9b95e25b80f65d39db9e26a9c53e5f8bb47b374ceb74101be68430fa60ce WatchSource:0}: Error finding container 5bfe9b95e25b80f65d39db9e26a9c53e5f8bb47b374ceb74101be68430fa60ce: Status 404 returned error can't find the container with id 5bfe9b95e25b80f65d39db9e26a9c53e5f8bb47b374ceb74101be68430fa60ce Sep 08 07:31:54.570835 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.570795 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:31:54.570915 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.570887 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:31:54.570954 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.570931 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:31:54.903752 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.903655 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" event={"ID":"6142a33b-fd87-4fca-8464-4f7604658f61","Type":"ContainerStarted","Data":"dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb"} Sep 08 07:31:54.903752 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.903711 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" event={"ID":"6142a33b-fd87-4fca-8464-4f7604658f61","Type":"ContainerStarted","Data":"5bfe9b95e25b80f65d39db9e26a9c53e5f8bb47b374ceb74101be68430fa60ce"} Sep 08 07:31:54.922914 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:54.922858 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" podStartSLOduration=1.922842817 podStartE2EDuration="1.922842817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:31:54.922307672 +0000 UTC m=+1629.600270825" watchObservedRunningTime="2026-09-08 07:31:54.922842817 +0000 UTC m=+1629.600805933" Sep 08 07:31:55.288406 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:55.288367 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:55.293458 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:55.293433 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:55.909958 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:55.909929 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:31:55.911015 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:31:55.910996 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:32:15.039988 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:15.039904 2571 generic.go:356] "Generic (PLEG): container finished" podID="6142a33b-fd87-4fca-8464-4f7604658f61" containerID="dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb" exitCode=0 Sep 08 07:32:15.040517 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:15.039982 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" event={"ID":"6142a33b-fd87-4fca-8464-4f7604658f61","Type":"ContainerDied","Data":"dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb"} Sep 08 07:32:15.040581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:15.040540 2571 scope.go:117] "RemoveContainer" containerID="dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb" Sep 08 07:32:15.041686 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:15.041656 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:32:15.041823 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:15.041740 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:32:15.041823 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:15.041789 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 08 07:32:16.047607 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:16.047570 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" event={"ID":"6142a33b-fd87-4fca-8464-4f7604658f61","Type":"ContainerStarted","Data":"d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51"} Sep 08 07:32:16.288099 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:16.288052 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:32:16.288441 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:16.288410 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Sep 08 07:32:16.288605 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:16.288484 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Sep 08 07:32:17.288294 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:17.288253 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Sep 08 07:32:17.288877 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:17.288844 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Sep 08 07:32:18.288236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:18.287662 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Sep 08 07:32:18.288236 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:18.287739 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Sep 08 07:32:19.288679 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:19.288638 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Sep 08 07:32:19.289292 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:19.289258 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Sep 08 07:32:20.288560 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:20.288521 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Sep 08 07:32:20.288876 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:20.288840 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Sep 08 07:32:21.291377 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:21.291345 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:32:21.291764 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:21.291601 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:32:21.292402 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:21.292380 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:32:26.849524 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:32:26.849487 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwl9h_9f88a82d-71c8-4d94-a9ff-e93e5b7f704c/payload-processing/0.log" Sep 08 07:33:02.512648 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:02.512615 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx_b86bc09a-26b7-462f-823c-260ad2f0d54e/payload-processing/0.log" Sep 08 07:33:02.631762 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:02.631726 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwl9h_9f88a82d-71c8-4d94-a9ff-e93e5b7f704c/payload-processing/0.log" Sep 08 07:33:08.385776 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.385696 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9"] Sep 08 07:33:08.388400 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.388378 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:08.390853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.390834 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-dd2695\"" Sep 08 07:33:08.399704 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.399676 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2"] Sep 08 07:33:08.402936 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.402906 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9"] Sep 08 07:33:08.403051 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.402950 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:08.405413 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.405394 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-dd2695-dockercfg-z82ms\"" Sep 08 07:33:08.411645 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.411609 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2"] Sep 08 07:33:08.509785 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.509746 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqmnh\" (UniqueName: \"kubernetes.io/projected/5a369ee4-420c-492f-a008-bdd9ebc861d7-kube-api-access-tqmnh\") pod \"payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9\" (UID: \"5a369ee4-420c-492f-a008-bdd9ebc861d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:08.509957 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.509801 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtkqt\" (UniqueName: \"kubernetes.io/projected/b46adaec-5842-404f-8d35-94b982ec0125-kube-api-access-vtkqt\") pod \"payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2\" (UID: \"b46adaec-5842-404f-8d35-94b982ec0125\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:08.509957 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.509879 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a369ee4-420c-492f-a008-bdd9ebc861d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9\" (UID: \"5a369ee4-420c-492f-a008-bdd9ebc861d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:08.510046 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.510016 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b46adaec-5842-404f-8d35-94b982ec0125-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2\" (UID: \"b46adaec-5842-404f-8d35-94b982ec0125\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:08.610812 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.610772 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b46adaec-5842-404f-8d35-94b982ec0125-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2\" (UID: \"b46adaec-5842-404f-8d35-94b982ec0125\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:08.611024 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.610842 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqmnh\" (UniqueName: \"kubernetes.io/projected/5a369ee4-420c-492f-a008-bdd9ebc861d7-kube-api-access-tqmnh\") pod \"payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9\" (UID: \"5a369ee4-420c-492f-a008-bdd9ebc861d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:08.611024 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.610878 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtkqt\" (UniqueName: \"kubernetes.io/projected/b46adaec-5842-404f-8d35-94b982ec0125-kube-api-access-vtkqt\") pod \"payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2\" (UID: \"b46adaec-5842-404f-8d35-94b982ec0125\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:08.611024 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.610901 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a369ee4-420c-492f-a008-bdd9ebc861d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9\" (UID: \"5a369ee4-420c-492f-a008-bdd9ebc861d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:08.611634 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.611609 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a369ee4-420c-492f-a008-bdd9ebc861d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9\" (UID: \"5a369ee4-420c-492f-a008-bdd9ebc861d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:08.611731 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.611611 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b46adaec-5842-404f-8d35-94b982ec0125-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2\" (UID: \"b46adaec-5842-404f-8d35-94b982ec0125\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:08.618979 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.618953 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtkqt\" (UniqueName: \"kubernetes.io/projected/b46adaec-5842-404f-8d35-94b982ec0125-kube-api-access-vtkqt\") pod \"payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2\" (UID: \"b46adaec-5842-404f-8d35-94b982ec0125\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:08.618979 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.618967 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqmnh\" (UniqueName: \"kubernetes.io/projected/5a369ee4-420c-492f-a008-bdd9ebc861d7-kube-api-access-tqmnh\") pod \"payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9\" (UID: \"5a369ee4-420c-492f-a008-bdd9ebc861d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:08.699672 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.699642 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:08.715194 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.715161 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:08.993917 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:08.993880 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9"] Sep 08 07:33:08.994838 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:33:08.994814 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a369ee4_420c_492f_a008_bdd9ebc861d7.slice/crio-4a16fed5c23d2655e04e8f648c9d7a76f695a550f269443c3894a265cf285b2e WatchSource:0}: Error finding container 4a16fed5c23d2655e04e8f648c9d7a76f695a550f269443c3894a265cf285b2e: Status 404 returned error can't find the container with id 4a16fed5c23d2655e04e8f648c9d7a76f695a550f269443c3894a265cf285b2e Sep 08 07:33:09.010098 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:09.010076 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2"] Sep 08 07:33:09.012061 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:33:09.012036 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb46adaec_5842_404f_8d35_94b982ec0125.slice/crio-a65b74d10d68b43e0b383a2b1fcaee61139f29d6d7ad750d0062120c29f0454d WatchSource:0}: Error finding container a65b74d10d68b43e0b383a2b1fcaee61139f29d6d7ad750d0062120c29f0454d: Status 404 returned error can't find the container with id a65b74d10d68b43e0b383a2b1fcaee61139f29d6d7ad750d0062120c29f0454d Sep 08 07:33:09.357181 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:09.357079 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" event={"ID":"b46adaec-5842-404f-8d35-94b982ec0125","Type":"ContainerStarted","Data":"62a2140b87f5c2730c51e670ff6a8aa27f6878b876bd4ea9af26e9ca59c444b0"} Sep 08 07:33:09.357181 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:09.357141 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" event={"ID":"b46adaec-5842-404f-8d35-94b982ec0125","Type":"ContainerStarted","Data":"a65b74d10d68b43e0b383a2b1fcaee61139f29d6d7ad750d0062120c29f0454d"} Sep 08 07:33:09.357426 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:09.357183 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:09.358779 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:09.358748 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" event={"ID":"5a369ee4-420c-492f-a008-bdd9ebc861d7","Type":"ContainerStarted","Data":"d8a023c20823e54b6dab5ea66d9ad7ae20438380d9c00429f7b587e7d2bcb2cc"} Sep 08 07:33:09.358905 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:09.358799 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" event={"ID":"5a369ee4-420c-492f-a008-bdd9ebc861d7","Type":"ContainerStarted","Data":"4a16fed5c23d2655e04e8f648c9d7a76f695a550f269443c3894a265cf285b2e"} Sep 08 07:33:09.358905 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:09.358824 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:09.379429 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:09.379363 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" podStartSLOduration=1.379345057 podStartE2EDuration="1.379345057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:33:09.378002458 +0000 UTC m=+1704.055965579" watchObservedRunningTime="2026-09-08 07:33:09.379345057 +0000 UTC m=+1704.057308178" Sep 08 07:33:09.397624 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:09.397564 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" podStartSLOduration=1.397543044 podStartE2EDuration="1.397543044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:33:09.394290367 +0000 UTC m=+1704.072253496" watchObservedRunningTime="2026-09-08 07:33:09.397543044 +0000 UTC m=+1704.075506163" Sep 08 07:33:20.367012 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:20.366979 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:20.367403 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:20.367150 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:32.105604 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:32.105567 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w"] Sep 08 07:33:32.109137 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:32.109120 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w" Sep 08 07:33:32.111558 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:32.111542 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-824nh\"" Sep 08 07:33:32.111810 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:32.111791 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 07:33:32.118529 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:32.118459 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w"] Sep 08 07:33:32.134788 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:32.134756 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb616d8-6b54-4d3a-ad58-626d80a74cd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w\" (UID: \"4bb616d8-6b54-4d3a-ad58-626d80a74cd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w" Sep 08 07:33:32.238650 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:32.238597 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb616d8-6b54-4d3a-ad58-626d80a74cd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w\" (UID: \"4bb616d8-6b54-4d3a-ad58-626d80a74cd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w" Sep 08 07:33:32.239527 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:32.239500 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb616d8-6b54-4d3a-ad58-626d80a74cd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w\" (UID: \"4bb616d8-6b54-4d3a-ad58-626d80a74cd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w" Sep 08 07:33:32.421592 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:32.421506 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w" Sep 08 07:33:32.693994 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:32.693963 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w"] Sep 08 07:33:32.695132 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:33:32.695102 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb616d8_6b54_4d3a_ad58_626d80a74cd0.slice/crio-a66c3de200291966a7502535bda73e59b2e735461756fd65747eedadc7c42042 WatchSource:0}: Error finding container a66c3de200291966a7502535bda73e59b2e735461756fd65747eedadc7c42042: Status 404 returned error can't find the container with id a66c3de200291966a7502535bda73e59b2e735461756fd65747eedadc7c42042 Sep 08 07:33:33.500621 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:33.500588 2571 generic.go:356] "Generic (PLEG): container finished" podID="4bb616d8-6b54-4d3a-ad58-626d80a74cd0" containerID="c750d86615f3fdd80c9a8f47c1739f8ff7016f70a934b9728b5a1195ee1b07d7" exitCode=0 Sep 08 07:33:33.501058 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:33.500678 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w" event={"ID":"4bb616d8-6b54-4d3a-ad58-626d80a74cd0","Type":"ContainerDied","Data":"c750d86615f3fdd80c9a8f47c1739f8ff7016f70a934b9728b5a1195ee1b07d7"} Sep 08 07:33:33.501058 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:33.500734 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w" event={"ID":"4bb616d8-6b54-4d3a-ad58-626d80a74cd0","Type":"ContainerStarted","Data":"a66c3de200291966a7502535bda73e59b2e735461756fd65747eedadc7c42042"} Sep 08 07:33:34.639450 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:34.639426 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w" Sep 08 07:33:34.661742 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:34.661721 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb616d8-6b54-4d3a-ad58-626d80a74cd0-maas-api-service-ca\") pod \"4bb616d8-6b54-4d3a-ad58-626d80a74cd0\" (UID: \"4bb616d8-6b54-4d3a-ad58-626d80a74cd0\") " Sep 08 07:33:34.662045 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:34.662024 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb616d8-6b54-4d3a-ad58-626d80a74cd0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4bb616d8-6b54-4d3a-ad58-626d80a74cd0" (UID: "4bb616d8-6b54-4d3a-ad58-626d80a74cd0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:33:34.763054 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:34.762956 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb616d8-6b54-4d3a-ad58-626d80a74cd0-maas-api-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:33:35.514420 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:35.514388 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w" Sep 08 07:33:35.514586 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:35.514422 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w" event={"ID":"4bb616d8-6b54-4d3a-ad58-626d80a74cd0","Type":"ContainerDied","Data":"a66c3de200291966a7502535bda73e59b2e735461756fd65747eedadc7c42042"} Sep 08 07:33:35.514586 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:35.514469 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a66c3de200291966a7502535bda73e59b2e735461756fd65747eedadc7c42042" Sep 08 07:33:38.259419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.259382 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2"] Sep 08 07:33:38.259890 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.259652 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" podUID="b46adaec-5842-404f-8d35-94b982ec0125" containerName="payload-processing" containerID="cri-o://62a2140b87f5c2730c51e670ff6a8aa27f6878b876bd4ea9af26e9ca59c444b0" gracePeriod=30 Sep 08 07:33:38.266661 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.266632 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9"] Sep 08 07:33:38.267504 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.267049 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" podUID="5a369ee4-420c-492f-a008-bdd9ebc861d7" containerName="payload-pre-processing" containerID="cri-o://d8a023c20823e54b6dab5ea66d9ad7ae20438380d9c00429f7b587e7d2bcb2cc" gracePeriod=30 Sep 08 07:33:38.438809 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.438774 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w"] Sep 08 07:33:38.441357 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.441330 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dd2695-n5c2w"] Sep 08 07:33:38.533543 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.533511 2571 generic.go:356] "Generic (PLEG): container finished" podID="5a369ee4-420c-492f-a008-bdd9ebc861d7" containerID="d8a023c20823e54b6dab5ea66d9ad7ae20438380d9c00429f7b587e7d2bcb2cc" exitCode=0 Sep 08 07:33:38.533674 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.533586 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" event={"ID":"5a369ee4-420c-492f-a008-bdd9ebc861d7","Type":"ContainerDied","Data":"d8a023c20823e54b6dab5ea66d9ad7ae20438380d9c00429f7b587e7d2bcb2cc"} Sep 08 07:33:38.535066 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.535037 2571 generic.go:356] "Generic (PLEG): container finished" podID="b46adaec-5842-404f-8d35-94b982ec0125" containerID="62a2140b87f5c2730c51e670ff6a8aa27f6878b876bd4ea9af26e9ca59c444b0" exitCode=0 Sep 08 07:33:38.535186 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.535083 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" event={"ID":"b46adaec-5842-404f-8d35-94b982ec0125","Type":"ContainerDied","Data":"62a2140b87f5c2730c51e670ff6a8aa27f6878b876bd4ea9af26e9ca59c444b0"} Sep 08 07:33:38.535186 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.535125 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" event={"ID":"b46adaec-5842-404f-8d35-94b982ec0125","Type":"ContainerDied","Data":"a65b74d10d68b43e0b383a2b1fcaee61139f29d6d7ad750d0062120c29f0454d"} Sep 08 07:33:38.535186 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.535148 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a65b74d10d68b43e0b383a2b1fcaee61139f29d6d7ad750d0062120c29f0454d" Sep 08 07:33:38.544702 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.544684 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:38.547308 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.547289 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:38.602060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.599240 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqmnh\" (UniqueName: \"kubernetes.io/projected/5a369ee4-420c-492f-a008-bdd9ebc861d7-kube-api-access-tqmnh\") pod \"5a369ee4-420c-492f-a008-bdd9ebc861d7\" (UID: \"5a369ee4-420c-492f-a008-bdd9ebc861d7\") " Sep 08 07:33:38.602060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.599339 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b46adaec-5842-404f-8d35-94b982ec0125-plugins-config-volume\") pod \"b46adaec-5842-404f-8d35-94b982ec0125\" (UID: \"b46adaec-5842-404f-8d35-94b982ec0125\") " Sep 08 07:33:38.602060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.599375 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a369ee4-420c-492f-a008-bdd9ebc861d7-plugins-config-volume\") pod \"5a369ee4-420c-492f-a008-bdd9ebc861d7\" (UID: \"5a369ee4-420c-492f-a008-bdd9ebc861d7\") " Sep 08 07:33:38.602060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.599403 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtkqt\" (UniqueName: \"kubernetes.io/projected/b46adaec-5842-404f-8d35-94b982ec0125-kube-api-access-vtkqt\") pod \"b46adaec-5842-404f-8d35-94b982ec0125\" (UID: \"b46adaec-5842-404f-8d35-94b982ec0125\") " Sep 08 07:33:38.602060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.600045 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b46adaec-5842-404f-8d35-94b982ec0125-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b46adaec-5842-404f-8d35-94b982ec0125" (UID: "b46adaec-5842-404f-8d35-94b982ec0125"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:33:38.602060 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.600439 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a369ee4-420c-492f-a008-bdd9ebc861d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5a369ee4-420c-492f-a008-bdd9ebc861d7" (UID: "5a369ee4-420c-492f-a008-bdd9ebc861d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:33:38.603936 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.603230 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b46adaec-5842-404f-8d35-94b982ec0125-kube-api-access-vtkqt" (OuterVolumeSpecName: "kube-api-access-vtkqt") pod "b46adaec-5842-404f-8d35-94b982ec0125" (UID: "b46adaec-5842-404f-8d35-94b982ec0125"). InnerVolumeSpecName "kube-api-access-vtkqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:33:38.618057 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.618003 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a369ee4-420c-492f-a008-bdd9ebc861d7-kube-api-access-tqmnh" (OuterVolumeSpecName: "kube-api-access-tqmnh") pod "5a369ee4-420c-492f-a008-bdd9ebc861d7" (UID: "5a369ee4-420c-492f-a008-bdd9ebc861d7"). InnerVolumeSpecName "kube-api-access-tqmnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:33:38.700625 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.700595 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqmnh\" (UniqueName: \"kubernetes.io/projected/5a369ee4-420c-492f-a008-bdd9ebc861d7-kube-api-access-tqmnh\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:33:38.700625 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.700620 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b46adaec-5842-404f-8d35-94b982ec0125-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:33:38.700625 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.700630 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a369ee4-420c-492f-a008-bdd9ebc861d7-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:33:38.700853 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:38.700638 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtkqt\" (UniqueName: \"kubernetes.io/projected/b46adaec-5842-404f-8d35-94b982ec0125-kube-api-access-vtkqt\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:33:39.542095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.542014 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" Sep 08 07:33:39.542095 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.542033 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9" event={"ID":"5a369ee4-420c-492f-a008-bdd9ebc861d7","Type":"ContainerDied","Data":"4a16fed5c23d2655e04e8f648c9d7a76f695a550f269443c3894a265cf285b2e"} Sep 08 07:33:39.542561 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.542100 2571 scope.go:117] "RemoveContainer" containerID="d8a023c20823e54b6dab5ea66d9ad7ae20438380d9c00429f7b587e7d2bcb2cc" Sep 08 07:33:39.542561 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.542286 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2" Sep 08 07:33:39.569775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.569745 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2"] Sep 08 07:33:39.572241 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.572203 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-dd2695-8dd7b6647-6n9v2"] Sep 08 07:33:39.582986 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.582918 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9"] Sep 08 07:33:39.585419 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.585397 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dd2695-665d756df8-bq2w9"] Sep 08 07:33:39.808989 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.808897 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bb616d8-6b54-4d3a-ad58-626d80a74cd0" path="/var/lib/kubelet/pods/4bb616d8-6b54-4d3a-ad58-626d80a74cd0/volumes" Sep 08 07:33:39.809410 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.809388 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a369ee4-420c-492f-a008-bdd9ebc861d7" path="/var/lib/kubelet/pods/5a369ee4-420c-492f-a008-bdd9ebc861d7/volumes" Sep 08 07:33:39.809780 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:39.809761 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b46adaec-5842-404f-8d35-94b982ec0125" path="/var/lib/kubelet/pods/b46adaec-5842-404f-8d35-94b982ec0125/volumes" Sep 08 07:33:48.967398 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.967359 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht"] Sep 08 07:33:48.968675 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.968642 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a369ee4-420c-492f-a008-bdd9ebc861d7" containerName="payload-pre-processing" Sep 08 07:33:48.968675 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.968676 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a369ee4-420c-492f-a008-bdd9ebc861d7" containerName="payload-pre-processing" Sep 08 07:33:48.968988 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.968726 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46adaec-5842-404f-8d35-94b982ec0125" containerName="payload-processing" Sep 08 07:33:48.968988 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.968744 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46adaec-5842-404f-8d35-94b982ec0125" containerName="payload-processing" Sep 08 07:33:48.968988 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.968763 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb616d8-6b54-4d3a-ad58-626d80a74cd0" containerName="revoke-keys" Sep 08 07:33:48.968988 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.968779 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb616d8-6b54-4d3a-ad58-626d80a74cd0" containerName="revoke-keys" Sep 08 07:33:48.968988 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.968947 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bb616d8-6b54-4d3a-ad58-626d80a74cd0" containerName="revoke-keys" Sep 08 07:33:48.968988 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.968960 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a369ee4-420c-492f-a008-bdd9ebc861d7" containerName="payload-pre-processing" Sep 08 07:33:48.968988 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.968977 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b46adaec-5842-404f-8d35-94b982ec0125" containerName="payload-processing" Sep 08 07:33:48.975929 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.975898 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht" Sep 08 07:33:48.979061 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.979035 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-824nh\"" Sep 08 07:33:48.979232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.979041 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 07:33:48.979232 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:48.979129 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht"] Sep 08 07:33:49.096200 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:49.096167 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14dd402b-dd8f-4e4f-a1dd-b01620ba3a72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht\" (UID: \"14dd402b-dd8f-4e4f-a1dd-b01620ba3a72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht" Sep 08 07:33:49.197204 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:49.197171 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14dd402b-dd8f-4e4f-a1dd-b01620ba3a72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht\" (UID: \"14dd402b-dd8f-4e4f-a1dd-b01620ba3a72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht" Sep 08 07:33:49.197830 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:49.197803 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14dd402b-dd8f-4e4f-a1dd-b01620ba3a72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht\" (UID: \"14dd402b-dd8f-4e4f-a1dd-b01620ba3a72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht" Sep 08 07:33:49.288831 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:49.288748 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht" Sep 08 07:33:49.562123 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:49.562098 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht"] Sep 08 07:33:49.564170 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:33:49.564140 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14dd402b_dd8f_4e4f_a1dd_b01620ba3a72.slice/crio-fc9487841c58b46d55e0a3c8d18295415d8937d7bc56e2e8991b4311f1427696 WatchSource:0}: Error finding container fc9487841c58b46d55e0a3c8d18295415d8937d7bc56e2e8991b4311f1427696: Status 404 returned error can't find the container with id fc9487841c58b46d55e0a3c8d18295415d8937d7bc56e2e8991b4311f1427696 Sep 08 07:33:49.601186 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:49.601162 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht" event={"ID":"14dd402b-dd8f-4e4f-a1dd-b01620ba3a72","Type":"ContainerStarted","Data":"fc9487841c58b46d55e0a3c8d18295415d8937d7bc56e2e8991b4311f1427696"} Sep 08 07:33:50.608634 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:50.608602 2571 generic.go:356] "Generic (PLEG): container finished" podID="14dd402b-dd8f-4e4f-a1dd-b01620ba3a72" containerID="35dcb64b6951b0f0777a3bad75ac17f17a090185d2a1f797c0c8fb023568b960" exitCode=0 Sep 08 07:33:50.609003 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:50.608692 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht" event={"ID":"14dd402b-dd8f-4e4f-a1dd-b01620ba3a72","Type":"ContainerDied","Data":"35dcb64b6951b0f0777a3bad75ac17f17a090185d2a1f797c0c8fb023568b960"} Sep 08 07:33:51.748751 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:51.748728 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht" Sep 08 07:33:51.821443 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:51.821414 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14dd402b-dd8f-4e4f-a1dd-b01620ba3a72-maas-api-service-ca\") pod \"14dd402b-dd8f-4e4f-a1dd-b01620ba3a72\" (UID: \"14dd402b-dd8f-4e4f-a1dd-b01620ba3a72\") " Sep 08 07:33:51.821795 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:51.821775 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14dd402b-dd8f-4e4f-a1dd-b01620ba3a72-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "14dd402b-dd8f-4e4f-a1dd-b01620ba3a72" (UID: "14dd402b-dd8f-4e4f-a1dd-b01620ba3a72"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:33:51.922996 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:51.922920 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14dd402b-dd8f-4e4f-a1dd-b01620ba3a72-maas-api-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:33:52.621857 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:52.621818 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht" event={"ID":"14dd402b-dd8f-4e4f-a1dd-b01620ba3a72","Type":"ContainerDied","Data":"fc9487841c58b46d55e0a3c8d18295415d8937d7bc56e2e8991b4311f1427696"} Sep 08 07:33:52.621857 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:52.621848 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht" Sep 08 07:33:52.622052 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:33:52.621879 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc9487841c58b46d55e0a3c8d18295415d8937d7bc56e2e8991b4311f1427696" Sep 08 07:34:04.625143 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:04.625103 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx"] Sep 08 07:34:04.626594 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:04.625474 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" podUID="b86bc09a-26b7-462f-823c-260ad2f0d54e" containerName="payload-processing" containerID="cri-o://9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8" gracePeriod=30 Sep 08 07:34:04.807179 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:04.807152 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht"] Sep 08 07:34:04.811376 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:04.811349 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4d980e-4spht"] Sep 08 07:34:04.880813 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:04.880752 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:34:05.049968 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.049934 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b86bc09a-26b7-462f-823c-260ad2f0d54e-plugins-config-volume\") pod \"b86bc09a-26b7-462f-823c-260ad2f0d54e\" (UID: \"b86bc09a-26b7-462f-823c-260ad2f0d54e\") " Sep 08 07:34:05.050162 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.050031 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2wj8\" (UniqueName: \"kubernetes.io/projected/b86bc09a-26b7-462f-823c-260ad2f0d54e-kube-api-access-b2wj8\") pod \"b86bc09a-26b7-462f-823c-260ad2f0d54e\" (UID: \"b86bc09a-26b7-462f-823c-260ad2f0d54e\") " Sep 08 07:34:05.050353 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.050324 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b86bc09a-26b7-462f-823c-260ad2f0d54e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b86bc09a-26b7-462f-823c-260ad2f0d54e" (UID: "b86bc09a-26b7-462f-823c-260ad2f0d54e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:34:05.052085 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.052065 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b86bc09a-26b7-462f-823c-260ad2f0d54e-kube-api-access-b2wj8" (OuterVolumeSpecName: "kube-api-access-b2wj8") pod "b86bc09a-26b7-462f-823c-260ad2f0d54e" (UID: "b86bc09a-26b7-462f-823c-260ad2f0d54e"). InnerVolumeSpecName "kube-api-access-b2wj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:34:05.151761 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.151664 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2wj8\" (UniqueName: \"kubernetes.io/projected/b86bc09a-26b7-462f-823c-260ad2f0d54e-kube-api-access-b2wj8\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:34:05.151761 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.151698 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b86bc09a-26b7-462f-823c-260ad2f0d54e-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:34:05.697544 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.697509 2571 generic.go:356] "Generic (PLEG): container finished" podID="b86bc09a-26b7-462f-823c-260ad2f0d54e" containerID="9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8" exitCode=0 Sep 08 07:34:05.698018 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.697557 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" Sep 08 07:34:05.698018 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.697599 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" event={"ID":"b86bc09a-26b7-462f-823c-260ad2f0d54e","Type":"ContainerDied","Data":"9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8"} Sep 08 07:34:05.698018 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.697660 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx" event={"ID":"b86bc09a-26b7-462f-823c-260ad2f0d54e","Type":"ContainerDied","Data":"28cf4cb4be9bf2511da7613b6506b1a130640ffdd517078bcce067c84d1ee241"} Sep 08 07:34:05.698018 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.697699 2571 scope.go:117] "RemoveContainer" containerID="9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8" Sep 08 07:34:05.707304 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.707286 2571 scope.go:117] "RemoveContainer" containerID="9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8" Sep 08 07:34:05.707568 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:34:05.707548 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8\": container with ID starting with 9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8 not found: ID does not exist" containerID="9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8" Sep 08 07:34:05.707614 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.707577 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8"} err="failed to get container status \"9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8\": rpc error: code = NotFound desc = could not find container \"9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8\": container with ID starting with 9715ace15f91a9346897cf10bb80d4d9a3be88f3c36325f5dcc0f8f6c254a7a8 not found: ID does not exist" Sep 08 07:34:05.720347 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.720307 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx"] Sep 08 07:34:05.724043 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.724024 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4d980e-9667c6889-tmtfx"] Sep 08 07:34:05.814302 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.814268 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14dd402b-dd8f-4e4f-a1dd-b01620ba3a72" path="/var/lib/kubelet/pods/14dd402b-dd8f-4e4f-a1dd-b01620ba3a72/volumes" Sep 08 07:34:05.814636 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:05.814624 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b86bc09a-26b7-462f-823c-260ad2f0d54e" path="/var/lib/kubelet/pods/b86bc09a-26b7-462f-823c-260ad2f0d54e/volumes" Sep 08 07:34:54.344665 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.344630 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq"] Sep 08 07:34:54.345108 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.345016 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b86bc09a-26b7-462f-823c-260ad2f0d54e" containerName="payload-processing" Sep 08 07:34:54.345108 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.345029 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b86bc09a-26b7-462f-823c-260ad2f0d54e" containerName="payload-processing" Sep 08 07:34:54.345108 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.345036 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14dd402b-dd8f-4e4f-a1dd-b01620ba3a72" containerName="revoke-keys" Sep 08 07:34:54.345108 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.345041 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="14dd402b-dd8f-4e4f-a1dd-b01620ba3a72" containerName="revoke-keys" Sep 08 07:34:54.345275 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.345116 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="14dd402b-dd8f-4e4f-a1dd-b01620ba3a72" containerName="revoke-keys" Sep 08 07:34:54.345275 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.345124 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b86bc09a-26b7-462f-823c-260ad2f0d54e" containerName="payload-processing" Sep 08 07:34:54.348183 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.348165 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq" Sep 08 07:34:54.350841 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.350813 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 07:34:54.350969 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.350813 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-824nh\"" Sep 08 07:34:54.355603 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.355506 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq"] Sep 08 07:34:54.411299 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.411257 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a205fb45-9566-4507-8690-6e0eefc50131-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq\" (UID: \"a205fb45-9566-4507-8690-6e0eefc50131\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq" Sep 08 07:34:54.511928 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.511892 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a205fb45-9566-4507-8690-6e0eefc50131-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq\" (UID: \"a205fb45-9566-4507-8690-6e0eefc50131\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq" Sep 08 07:34:54.512577 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.512558 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a205fb45-9566-4507-8690-6e0eefc50131-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq\" (UID: \"a205fb45-9566-4507-8690-6e0eefc50131\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq" Sep 08 07:34:54.659411 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.659305 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq" Sep 08 07:34:54.947334 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.947304 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq"] Sep 08 07:34:54.948137 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:34:54.948114 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda205fb45_9566_4507_8690_6e0eefc50131.slice/crio-14b24795575e4c60b97c68a02c925e85badffc8cd4907e08af26460a1b3bdf4f WatchSource:0}: Error finding container 14b24795575e4c60b97c68a02c925e85badffc8cd4907e08af26460a1b3bdf4f: Status 404 returned error can't find the container with id 14b24795575e4c60b97c68a02c925e85badffc8cd4907e08af26460a1b3bdf4f Sep 08 07:34:54.988939 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:54.988910 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq" event={"ID":"a205fb45-9566-4507-8690-6e0eefc50131","Type":"ContainerStarted","Data":"14b24795575e4c60b97c68a02c925e85badffc8cd4907e08af26460a1b3bdf4f"} Sep 08 07:34:55.995827 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:55.995788 2571 generic.go:356] "Generic (PLEG): container finished" podID="a205fb45-9566-4507-8690-6e0eefc50131" containerID="21d320cec589e6cc211ae89d9d79380bf43216ecb0b58374227b73440a128551" exitCode=0 Sep 08 07:34:55.996203 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:55.995876 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq" event={"ID":"a205fb45-9566-4507-8690-6e0eefc50131","Type":"ContainerDied","Data":"21d320cec589e6cc211ae89d9d79380bf43216ecb0b58374227b73440a128551"} Sep 08 07:34:57.130190 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:57.130166 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq" Sep 08 07:34:57.238517 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:57.238473 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a205fb45-9566-4507-8690-6e0eefc50131-maas-api-service-ca\") pod \"a205fb45-9566-4507-8690-6e0eefc50131\" (UID: \"a205fb45-9566-4507-8690-6e0eefc50131\") " Sep 08 07:34:57.238901 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:57.238875 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a205fb45-9566-4507-8690-6e0eefc50131-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a205fb45-9566-4507-8690-6e0eefc50131" (UID: "a205fb45-9566-4507-8690-6e0eefc50131"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:34:57.339890 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:57.339804 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a205fb45-9566-4507-8690-6e0eefc50131-maas-api-service-ca\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:34:58.008291 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:58.008257 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq" Sep 08 07:34:58.008291 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:58.008277 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq" event={"ID":"a205fb45-9566-4507-8690-6e0eefc50131","Type":"ContainerDied","Data":"14b24795575e4c60b97c68a02c925e85badffc8cd4907e08af26460a1b3bdf4f"} Sep 08 07:34:58.008508 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:34:58.008331 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14b24795575e4c60b97c68a02c925e85badffc8cd4907e08af26460a1b3bdf4f" Sep 08 07:35:05.159499 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:05.159459 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq"] Sep 08 07:35:05.161953 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:05.161929 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-79da86-hqndq"] Sep 08 07:35:05.809761 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:05.809726 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a205fb45-9566-4507-8690-6e0eefc50131" path="/var/lib/kubelet/pods/a205fb45-9566-4507-8690-6e0eefc50131/volumes" Sep 08 07:35:12.397587 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:12.397551 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz"] Sep 08 07:35:12.398002 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:12.397773 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" containerID="cri-o://d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51" gracePeriod=30 Sep 08 07:35:17.641040 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.641010 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:35:17.739939 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.739908 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6142a33b-fd87-4fca-8464-4f7604658f61-istio-podinfo\") pod \"6142a33b-fd87-4fca-8464-4f7604658f61\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " Sep 08 07:35:17.740129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.739955 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-certs\") pod \"6142a33b-fd87-4fca-8464-4f7604658f61\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " Sep 08 07:35:17.740129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.739998 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6142a33b-fd87-4fca-8464-4f7604658f61-istiod-ca-cert\") pod \"6142a33b-fd87-4fca-8464-4f7604658f61\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " Sep 08 07:35:17.740129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740020 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-socket\") pod \"6142a33b-fd87-4fca-8464-4f7604658f61\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " Sep 08 07:35:17.740129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740037 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-data\") pod \"6142a33b-fd87-4fca-8464-4f7604658f61\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " Sep 08 07:35:17.740129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740060 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-credential-socket\") pod \"6142a33b-fd87-4fca-8464-4f7604658f61\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " Sep 08 07:35:17.740129 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740109 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmhst\" (UniqueName: \"kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-kube-api-access-nmhst\") pod \"6142a33b-fd87-4fca-8464-4f7604658f61\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " Sep 08 07:35:17.740503 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740132 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-envoy\") pod \"6142a33b-fd87-4fca-8464-4f7604658f61\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " Sep 08 07:35:17.740503 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740165 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-istio-token\") pod \"6142a33b-fd87-4fca-8464-4f7604658f61\" (UID: \"6142a33b-fd87-4fca-8464-4f7604658f61\") " Sep 08 07:35:17.740503 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740410 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6142a33b-fd87-4fca-8464-4f7604658f61" (UID: "6142a33b-fd87-4fca-8464-4f7604658f61"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:35:17.740503 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740421 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6142a33b-fd87-4fca-8464-4f7604658f61" (UID: "6142a33b-fd87-4fca-8464-4f7604658f61"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:35:17.740503 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740433 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-data" (OuterVolumeSpecName: "istio-data") pod "6142a33b-fd87-4fca-8464-4f7604658f61" (UID: "6142a33b-fd87-4fca-8464-4f7604658f61"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:35:17.740798 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740514 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6142a33b-fd87-4fca-8464-4f7604658f61-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6142a33b-fd87-4fca-8464-4f7604658f61" (UID: "6142a33b-fd87-4fca-8464-4f7604658f61"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:35:17.740798 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740551 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6142a33b-fd87-4fca-8464-4f7604658f61" (UID: "6142a33b-fd87-4fca-8464-4f7604658f61"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:35:17.740798 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740591 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:17.740798 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740609 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6142a33b-fd87-4fca-8464-4f7604658f61-istiod-ca-cert\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:17.740798 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740622 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-workload-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:17.740798 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.740637 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-data\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:17.742384 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.742359 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-kube-api-access-nmhst" (OuterVolumeSpecName: "kube-api-access-nmhst") pod "6142a33b-fd87-4fca-8464-4f7604658f61" (UID: "6142a33b-fd87-4fca-8464-4f7604658f61"). InnerVolumeSpecName "kube-api-access-nmhst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:35:17.742460 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.742446 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-istio-token" (OuterVolumeSpecName: "istio-token") pod "6142a33b-fd87-4fca-8464-4f7604658f61" (UID: "6142a33b-fd87-4fca-8464-4f7604658f61"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:35:17.742540 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.742521 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6142a33b-fd87-4fca-8464-4f7604658f61-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6142a33b-fd87-4fca-8464-4f7604658f61" (UID: "6142a33b-fd87-4fca-8464-4f7604658f61"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 07:35:17.742693 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.742672 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6142a33b-fd87-4fca-8464-4f7604658f61" (UID: "6142a33b-fd87-4fca-8464-4f7604658f61"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:35:17.841975 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.841945 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmhst\" (UniqueName: \"kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-kube-api-access-nmhst\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:17.841975 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.841974 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-istio-envoy\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:17.842166 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.841983 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6142a33b-fd87-4fca-8464-4f7604658f61-istio-token\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:17.842166 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.841991 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6142a33b-fd87-4fca-8464-4f7604658f61-istio-podinfo\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:17.842166 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:17.841999 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6142a33b-fd87-4fca-8464-4f7604658f61-credential-socket\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:18.124811 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.124719 2571 generic.go:356] "Generic (PLEG): container finished" podID="6142a33b-fd87-4fca-8464-4f7604658f61" containerID="d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51" exitCode=0 Sep 08 07:35:18.124811 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.124759 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" event={"ID":"6142a33b-fd87-4fca-8464-4f7604658f61","Type":"ContainerDied","Data":"d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51"} Sep 08 07:35:18.124811 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.124771 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" Sep 08 07:35:18.124811 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.124803 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz" event={"ID":"6142a33b-fd87-4fca-8464-4f7604658f61","Type":"ContainerDied","Data":"5bfe9b95e25b80f65d39db9e26a9c53e5f8bb47b374ceb74101be68430fa60ce"} Sep 08 07:35:18.125114 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.124849 2571 scope.go:117] "RemoveContainer" containerID="d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51" Sep 08 07:35:18.134282 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.134262 2571 scope.go:117] "RemoveContainer" containerID="dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb" Sep 08 07:35:18.142678 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.142662 2571 scope.go:117] "RemoveContainer" containerID="d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51" Sep 08 07:35:18.142941 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:35:18.142916 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51\": container with ID starting with d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51 not found: ID does not exist" containerID="d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51" Sep 08 07:35:18.143015 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.142941 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51"} err="failed to get container status \"d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51\": rpc error: code = NotFound desc = could not find container \"d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51\": container with ID starting with d4fee270288e67ca429ab77257975f3ab0f80dd3816152416db7650ef1b29a51 not found: ID does not exist" Sep 08 07:35:18.143015 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.142956 2571 scope.go:117] "RemoveContainer" containerID="dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb" Sep 08 07:35:18.143293 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:35:18.143199 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb\": container with ID starting with dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb not found: ID does not exist" containerID="dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb" Sep 08 07:35:18.143293 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.143280 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb"} err="failed to get container status \"dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb\": rpc error: code = NotFound desc = could not find container \"dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb\": container with ID starting with dd0f69aa171554869832021137d0d257268a9cbc5aa800f0d833790baeaa32eb not found: ID does not exist" Sep 08 07:35:18.152990 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.152965 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz"] Sep 08 07:35:18.156924 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:18.156900 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-79da86-openshift-default-77bc9c6d94-l8tbz"] Sep 08 07:35:19.809561 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:19.809530 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" path="/var/lib/kubelet/pods/6142a33b-fd87-4fca-8464-4f7604658f61/volumes" Sep 08 07:35:23.362892 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.362855 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb"] Sep 08 07:35:23.365155 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.365131 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-fb59c2-29814210-lpnpb"] Sep 08 07:35:23.392640 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.392604 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7"] Sep 08 07:35:23.392910 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.392879 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" podUID="3b201b98-b85e-480f-bc41-0c8729533142" containerName="payload-processing" containerID="cri-o://f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e" gracePeriod=30 Sep 08 07:35:23.644286 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.644258 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:35:23.799386 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.799349 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p98np\" (UniqueName: \"kubernetes.io/projected/3b201b98-b85e-480f-bc41-0c8729533142-kube-api-access-p98np\") pod \"3b201b98-b85e-480f-bc41-0c8729533142\" (UID: \"3b201b98-b85e-480f-bc41-0c8729533142\") " Sep 08 07:35:23.799583 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.799453 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b201b98-b85e-480f-bc41-0c8729533142-plugins-config-volume\") pod \"3b201b98-b85e-480f-bc41-0c8729533142\" (UID: \"3b201b98-b85e-480f-bc41-0c8729533142\") " Sep 08 07:35:23.799815 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.799789 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b201b98-b85e-480f-bc41-0c8729533142-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b201b98-b85e-480f-bc41-0c8729533142" (UID: "3b201b98-b85e-480f-bc41-0c8729533142"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:35:23.801552 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.801529 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b201b98-b85e-480f-bc41-0c8729533142-kube-api-access-p98np" (OuterVolumeSpecName: "kube-api-access-p98np") pod "3b201b98-b85e-480f-bc41-0c8729533142" (UID: "3b201b98-b85e-480f-bc41-0c8729533142"). InnerVolumeSpecName "kube-api-access-p98np". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:35:23.809087 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.809061 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72eb1972-dbd3-4fbc-a655-d44035eac274" path="/var/lib/kubelet/pods/72eb1972-dbd3-4fbc-a655-d44035eac274/volumes" Sep 08 07:35:23.900606 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.900519 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p98np\" (UniqueName: \"kubernetes.io/projected/3b201b98-b85e-480f-bc41-0c8729533142-kube-api-access-p98np\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:23.900606 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:23.900547 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b201b98-b85e-480f-bc41-0c8729533142-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:24.161559 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:24.161465 2571 generic.go:356] "Generic (PLEG): container finished" podID="3b201b98-b85e-480f-bc41-0c8729533142" containerID="f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e" exitCode=0 Sep 08 07:35:24.161559 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:24.161536 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" Sep 08 07:35:24.161801 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:24.161555 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" event={"ID":"3b201b98-b85e-480f-bc41-0c8729533142","Type":"ContainerDied","Data":"f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e"} Sep 08 07:35:24.161801 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:24.161622 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7" event={"ID":"3b201b98-b85e-480f-bc41-0c8729533142","Type":"ContainerDied","Data":"8f64f68b2b62a9e11fb5d153aaf53205d0d5daecab70618719367b88867d1259"} Sep 08 07:35:24.161801 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:24.161651 2571 scope.go:117] "RemoveContainer" containerID="f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e" Sep 08 07:35:24.170711 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:24.170692 2571 scope.go:117] "RemoveContainer" containerID="f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e" Sep 08 07:35:24.171004 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:35:24.170982 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e\": container with ID starting with f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e not found: ID does not exist" containerID="f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e" Sep 08 07:35:24.171047 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:24.171012 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e"} err="failed to get container status \"f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e\": rpc error: code = NotFound desc = could not find container \"f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e\": container with ID starting with f50f27044f9ca4be3fffca0ac5c707fbb2ba314be3a85fd8f20b54d1871f8c4e not found: ID does not exist" Sep 08 07:35:24.182697 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:24.182670 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7"] Sep 08 07:35:24.185161 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:24.185134 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-fb59c2-5b99f64c6f-tjvn7"] Sep 08 07:35:25.809518 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:25.809486 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b201b98-b85e-480f-bc41-0c8729533142" path="/var/lib/kubelet/pods/3b201b98-b85e-480f-bc41-0c8729533142/volumes" Sep 08 07:35:42.881465 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:42.881432 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm"] Sep 08 07:35:42.881902 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:42.881648 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" podUID="03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8" containerName="payload-processing" containerID="cri-o://8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45" gracePeriod=30 Sep 08 07:35:42.901179 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:42.901150 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj"] Sep 08 07:35:42.901491 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:42.901453 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" podUID="74eb1128-c920-4f17-8e22-f335eb5d4010" containerName="payload-pre-processing" containerID="cri-o://f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a" gracePeriod=30 Sep 08 07:35:43.172452 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.172430 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:35:43.175159 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.175133 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:35:43.260792 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.260755 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4vlc\" (UniqueName: \"kubernetes.io/projected/74eb1128-c920-4f17-8e22-f335eb5d4010-kube-api-access-z4vlc\") pod \"74eb1128-c920-4f17-8e22-f335eb5d4010\" (UID: \"74eb1128-c920-4f17-8e22-f335eb5d4010\") " Sep 08 07:35:43.260959 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.260839 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-plugins-config-volume\") pod \"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8\" (UID: \"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8\") " Sep 08 07:35:43.260959 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.260870 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6blhr\" (UniqueName: \"kubernetes.io/projected/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-kube-api-access-6blhr\") pod \"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8\" (UID: \"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8\") " Sep 08 07:35:43.260959 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.260896 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74eb1128-c920-4f17-8e22-f335eb5d4010-plugins-config-volume\") pod \"74eb1128-c920-4f17-8e22-f335eb5d4010\" (UID: \"74eb1128-c920-4f17-8e22-f335eb5d4010\") " Sep 08 07:35:43.261311 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.261285 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8" (UID: "03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:35:43.261437 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.261355 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74eb1128-c920-4f17-8e22-f335eb5d4010-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "74eb1128-c920-4f17-8e22-f335eb5d4010" (UID: "74eb1128-c920-4f17-8e22-f335eb5d4010"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 07:35:43.263535 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.263506 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74eb1128-c920-4f17-8e22-f335eb5d4010-kube-api-access-z4vlc" (OuterVolumeSpecName: "kube-api-access-z4vlc") pod "74eb1128-c920-4f17-8e22-f335eb5d4010" (UID: "74eb1128-c920-4f17-8e22-f335eb5d4010"). InnerVolumeSpecName "kube-api-access-z4vlc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:35:43.263635 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.263515 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-kube-api-access-6blhr" (OuterVolumeSpecName: "kube-api-access-6blhr") pod "03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8" (UID: "03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8"). InnerVolumeSpecName "kube-api-access-6blhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:35:43.267517 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.267487 2571 generic.go:356] "Generic (PLEG): container finished" podID="03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8" containerID="8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45" exitCode=0 Sep 08 07:35:43.267618 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.267531 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" Sep 08 07:35:43.267618 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.267567 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" event={"ID":"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8","Type":"ContainerDied","Data":"8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45"} Sep 08 07:35:43.267618 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.267613 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm" event={"ID":"03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8","Type":"ContainerDied","Data":"3ed9d89476874b999581939dc2a67776901edcd686bdec1fdd3b3ff88340ddeb"} Sep 08 07:35:43.267749 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.267636 2571 scope.go:117] "RemoveContainer" containerID="8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45" Sep 08 07:35:43.269008 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.268986 2571 generic.go:356] "Generic (PLEG): container finished" podID="74eb1128-c920-4f17-8e22-f335eb5d4010" containerID="f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a" exitCode=0 Sep 08 07:35:43.269102 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.269014 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" event={"ID":"74eb1128-c920-4f17-8e22-f335eb5d4010","Type":"ContainerDied","Data":"f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a"} Sep 08 07:35:43.269102 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.269036 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" Sep 08 07:35:43.269102 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.269063 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj" event={"ID":"74eb1128-c920-4f17-8e22-f335eb5d4010","Type":"ContainerDied","Data":"1507c85e9410de1c0daf1321ef44698723b50475be9a8a6fe8042cdd2ee4cd8d"} Sep 08 07:35:43.277359 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.277100 2571 scope.go:117] "RemoveContainer" containerID="8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45" Sep 08 07:35:43.277493 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:35:43.277466 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45\": container with ID starting with 8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45 not found: ID does not exist" containerID="8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45" Sep 08 07:35:43.277559 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.277504 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45"} err="failed to get container status \"8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45\": rpc error: code = NotFound desc = could not find container \"8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45\": container with ID starting with 8fc13d0baca488fba60e81e392e64185354971603f00ce33538f68679f67fb45 not found: ID does not exist" Sep 08 07:35:43.277559 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.277526 2571 scope.go:117] "RemoveContainer" containerID="f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a" Sep 08 07:35:43.285791 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.285756 2571 scope.go:117] "RemoveContainer" containerID="f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a" Sep 08 07:35:43.286022 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:35:43.286007 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a\": container with ID starting with f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a not found: ID does not exist" containerID="f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a" Sep 08 07:35:43.286070 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.286026 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a"} err="failed to get container status \"f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a\": rpc error: code = NotFound desc = could not find container \"f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a\": container with ID starting with f54fe8f22ff6d0b7e3a470fa6b6aea5710862c5b72076e21cd4dc6d7510a493a not found: ID does not exist" Sep 08 07:35:43.298994 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.298971 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm"] Sep 08 07:35:43.302905 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.302886 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-217bc2-84958597dc-d4nzm"] Sep 08 07:35:43.316226 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.316193 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj"] Sep 08 07:35:43.327034 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.327013 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-217bc2-74fddd77d6-j46dj"] Sep 08 07:35:43.362356 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.362333 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4vlc\" (UniqueName: \"kubernetes.io/projected/74eb1128-c920-4f17-8e22-f335eb5d4010-kube-api-access-z4vlc\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:43.362356 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.362356 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:43.362506 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.362366 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6blhr\" (UniqueName: \"kubernetes.io/projected/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8-kube-api-access-6blhr\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:43.362506 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.362374 2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74eb1128-c920-4f17-8e22-f335eb5d4010-plugins-config-volume\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:35:43.810009 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.809967 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8" path="/var/lib/kubelet/pods/03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8/volumes" Sep 08 07:35:43.810446 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:35:43.810425 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74eb1128-c920-4f17-8e22-f335eb5d4010" path="/var/lib/kubelet/pods/74eb1128-c920-4f17-8e22-f335eb5d4010/volumes" Sep 08 07:36:46.157975 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:36:46.157941 2571 scope.go:117] "RemoveContainer" containerID="227eb863493f032f8ec19eede434b6ac238fd7336d3668f23c3d64f1f0d91938" Sep 08 07:36:46.166908 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:36:46.166888 2571 scope.go:117] "RemoveContainer" containerID="45910516a97b25fa41fdfa78eb6fc4dc6ce953b00f4dac764855cceece1a796d" Sep 08 07:39:46.208140 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:46.208109 2571 scope.go:117] "RemoveContainer" containerID="c750d86615f3fdd80c9a8f47c1739f8ff7016f70a934b9728b5a1195ee1b07d7" Sep 08 07:39:46.216976 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:46.216952 2571 scope.go:117] "RemoveContainer" containerID="62a2140b87f5c2730c51e670ff6a8aa27f6878b876bd4ea9af26e9ca59c444b0" Sep 08 07:39:59.153685 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.153644 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j"] Sep 08 07:39:59.154048 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154002 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b201b98-b85e-480f-bc41-0c8729533142" containerName="payload-processing" Sep 08 07:39:59.154048 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154016 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b201b98-b85e-480f-bc41-0c8729533142" containerName="payload-processing" Sep 08 07:39:59.154048 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154029 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74eb1128-c920-4f17-8e22-f335eb5d4010" containerName="payload-pre-processing" Sep 08 07:39:59.154048 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154037 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="74eb1128-c920-4f17-8e22-f335eb5d4010" containerName="payload-pre-processing" Sep 08 07:39:59.154048 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154048 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8" containerName="payload-processing" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154057 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8" containerName="payload-processing" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154066 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154075 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154088 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154097 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154131 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a205fb45-9566-4507-8690-6e0eefc50131" containerName="revoke-keys" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154140 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a205fb45-9566-4507-8690-6e0eefc50131" containerName="revoke-keys" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154253 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154269 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="74eb1128-c920-4f17-8e22-f335eb5d4010" containerName="payload-pre-processing" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154279 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6142a33b-fd87-4fca-8464-4f7604658f61" containerName="istio-proxy" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154286 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b201b98-b85e-480f-bc41-0c8729533142" containerName="payload-processing" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154295 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="03125a9d-ba1a-48af-93a9-2e8f7a6c9bd8" containerName="payload-processing" Sep 08 07:39:59.154314 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.154301 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a205fb45-9566-4507-8690-6e0eefc50131" containerName="revoke-keys" Sep 08 07:39:59.157501 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.157487 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.159833 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.159816 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-220f0417-kserve-self-signed-certs\"" Sep 08 07:39:59.166787 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.166766 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j"] Sep 08 07:39:59.225131 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.225100 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-home\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.225332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.225147 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-dshm\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.225332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.225229 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-model-cache\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.225332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.225268 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2xkb\" (UniqueName: \"kubernetes.io/projected/b9a2ce0e-57ba-482d-a095-cfce4484e784-kube-api-access-f2xkb\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.225332 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.225300 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a2ce0e-57ba-482d-a095-cfce4484e784-tls-certs\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.225532 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.225377 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-tmp-dir\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.326631 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.326584 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-dshm\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.326822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.326642 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-model-cache\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.326822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.326674 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2xkb\" (UniqueName: \"kubernetes.io/projected/b9a2ce0e-57ba-482d-a095-cfce4484e784-kube-api-access-f2xkb\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.326822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.326711 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a2ce0e-57ba-482d-a095-cfce4484e784-tls-certs\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.326822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.326777 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-tmp-dir\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.326822 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.326817 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-home\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.327096 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.327076 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-model-cache\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.327157 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.327114 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-home\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.327352 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.327331 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-tmp-dir\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.329081 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.329044 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-dshm\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.329359 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.329343 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a2ce0e-57ba-482d-a095-cfce4484e784-tls-certs\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.336448 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.336427 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2xkb\" (UniqueName: \"kubernetes.io/projected/b9a2ce0e-57ba-482d-a095-cfce4484e784-kube-api-access-f2xkb\") pod \"e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.467459 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.467428 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:39:59.740239 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:39:59.740197 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j"] Sep 08 07:39:59.742207 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:39:59.742182 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9a2ce0e_57ba_482d_a095_cfce4484e784.slice/crio-ce76692a8a9ab2d54271c54e14013e903b245fbb76b62e8e240ba0f9f206391c WatchSource:0}: Error finding container ce76692a8a9ab2d54271c54e14013e903b245fbb76b62e8e240ba0f9f206391c: Status 404 returned error can't find the container with id ce76692a8a9ab2d54271c54e14013e903b245fbb76b62e8e240ba0f9f206391c Sep 08 07:40:00.666537 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:40:00.666497 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" event={"ID":"b9a2ce0e-57ba-482d-a095-cfce4484e784","Type":"ContainerStarted","Data":"2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434"} Sep 08 07:40:00.666537 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:40:00.666543 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" event={"ID":"b9a2ce0e-57ba-482d-a095-cfce4484e784","Type":"ContainerStarted","Data":"ce76692a8a9ab2d54271c54e14013e903b245fbb76b62e8e240ba0f9f206391c"} Sep 08 07:40:00.684114 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:40:00.684062 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" podStartSLOduration=1.6840466680000001 podStartE2EDuration="1.684046668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 07:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:40:00.682751394 +0000 UTC m=+2115.360714508" watchObservedRunningTime="2026-09-08 07:40:00.684046668 +0000 UTC m=+2115.362009828" Sep 08 07:40:09.467852 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:40:09.467818 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:40:09.468272 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:40:09.467893 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:40:09.480301 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:40:09.480278 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:40:09.726235 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:40:09.726140 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:40:46.243758 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:40:46.243719 2571 scope.go:117] "RemoveContainer" containerID="35dcb64b6951b0f0777a3bad75ac17f17a090185d2a1f797c0c8fb023568b960" Sep 08 07:40:50.424575 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:40:50.424538 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j"] Sep 08 07:41:05.444341 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.444272 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" podUID="b9a2ce0e-57ba-482d-a095-cfce4484e784" containerName="main" containerID="cri-o://2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434" gracePeriod=45 Sep 08 07:41:05.698324 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.698263 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:41:05.849960 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.849922 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-tmp-dir\") pod \"b9a2ce0e-57ba-482d-a095-cfce4484e784\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " Sep 08 07:41:05.850153 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.849994 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-dshm\") pod \"b9a2ce0e-57ba-482d-a095-cfce4484e784\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " Sep 08 07:41:05.850153 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.850014 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-model-cache\") pod \"b9a2ce0e-57ba-482d-a095-cfce4484e784\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " Sep 08 07:41:05.850153 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.850030 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a2ce0e-57ba-482d-a095-cfce4484e784-tls-certs\") pod \"b9a2ce0e-57ba-482d-a095-cfce4484e784\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " Sep 08 07:41:05.850153 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.850052 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2xkb\" (UniqueName: \"kubernetes.io/projected/b9a2ce0e-57ba-482d-a095-cfce4484e784-kube-api-access-f2xkb\") pod \"b9a2ce0e-57ba-482d-a095-cfce4484e784\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " Sep 08 07:41:05.850457 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.850179 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-home\") pod \"b9a2ce0e-57ba-482d-a095-cfce4484e784\" (UID: \"b9a2ce0e-57ba-482d-a095-cfce4484e784\") " Sep 08 07:41:05.850457 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.850269 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b9a2ce0e-57ba-482d-a095-cfce4484e784" (UID: "b9a2ce0e-57ba-482d-a095-cfce4484e784"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:41:05.850457 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.850361 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-model-cache" (OuterVolumeSpecName: "model-cache") pod "b9a2ce0e-57ba-482d-a095-cfce4484e784" (UID: "b9a2ce0e-57ba-482d-a095-cfce4484e784"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:41:05.850457 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.850414 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-home" (OuterVolumeSpecName: "home") pod "b9a2ce0e-57ba-482d-a095-cfce4484e784" (UID: "b9a2ce0e-57ba-482d-a095-cfce4484e784"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:41:05.850679 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.850569 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-tmp-dir\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:41:05.850679 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.850582 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-model-cache\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:41:05.850679 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.850591 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-home\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:41:05.852369 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.852346 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9a2ce0e-57ba-482d-a095-cfce4484e784-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b9a2ce0e-57ba-482d-a095-cfce4484e784" (UID: "b9a2ce0e-57ba-482d-a095-cfce4484e784"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 07:41:05.852471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.852369 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-dshm" (OuterVolumeSpecName: "dshm") pod "b9a2ce0e-57ba-482d-a095-cfce4484e784" (UID: "b9a2ce0e-57ba-482d-a095-cfce4484e784"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 07:41:05.852471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.852440 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9a2ce0e-57ba-482d-a095-cfce4484e784-kube-api-access-f2xkb" (OuterVolumeSpecName: "kube-api-access-f2xkb") pod "b9a2ce0e-57ba-482d-a095-cfce4484e784" (UID: "b9a2ce0e-57ba-482d-a095-cfce4484e784"). InnerVolumeSpecName "kube-api-access-f2xkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 07:41:05.951471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.951390 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9a2ce0e-57ba-482d-a095-cfce4484e784-dshm\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:41:05.951471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.951417 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a2ce0e-57ba-482d-a095-cfce4484e784-tls-certs\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:41:05.951471 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:05.951428 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2xkb\" (UniqueName: \"kubernetes.io/projected/b9a2ce0e-57ba-482d-a095-cfce4484e784-kube-api-access-f2xkb\") on node \"ip-10-0-142-34.ec2.internal\" DevicePath \"\"" Sep 08 07:41:06.018975 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:06.018937 2571 generic.go:356] "Generic (PLEG): container finished" podID="b9a2ce0e-57ba-482d-a095-cfce4484e784" containerID="2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434" exitCode=0 Sep 08 07:41:06.018975 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:06.018979 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" Sep 08 07:41:06.019191 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:06.019016 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" event={"ID":"b9a2ce0e-57ba-482d-a095-cfce4484e784","Type":"ContainerDied","Data":"2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434"} Sep 08 07:41:06.019191 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:06.019063 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j" event={"ID":"b9a2ce0e-57ba-482d-a095-cfce4484e784","Type":"ContainerDied","Data":"ce76692a8a9ab2d54271c54e14013e903b245fbb76b62e8e240ba0f9f206391c"} Sep 08 07:41:06.019191 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:06.019081 2571 scope.go:117] "RemoveContainer" containerID="2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434" Sep 08 07:41:06.029092 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:06.029070 2571 scope.go:117] "RemoveContainer" containerID="2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434" Sep 08 07:41:06.029417 ip-10-0-142-34 kubenswrapper[2571]: E0908 07:41:06.029395 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434\": container with ID starting with 2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434 not found: ID does not exist" containerID="2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434" Sep 08 07:41:06.029506 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:06.029425 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434"} err="failed to get container status \"2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434\": rpc error: code = NotFound desc = could not find container \"2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434\": container with ID starting with 2215906ba7c7f6fb4f3ecb41e80de904a1c82f5163effb09712ba284c6ee9434 not found: ID does not exist" Sep 08 07:41:06.041784 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:06.041761 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j"] Sep 08 07:41:06.047728 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:06.047703 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-220f0417-kserve-76666f7455-h6h9j"] Sep 08 07:41:07.809581 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:07.809547 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9a2ce0e-57ba-482d-a095-cfce4484e784" path="/var/lib/kubelet/pods/b9a2ce0e-57ba-482d-a095-cfce4484e784/volumes" Sep 08 07:41:46.268407 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:46.268374 2571 scope.go:117] "RemoveContainer" containerID="21d320cec589e6cc211ae89d9d79380bf43216ecb0b58374227b73440a128551" Sep 08 07:41:50.591779 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:50.591751 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6qbws_d4b16240-e668-43ca-a967-2649668f1951/server/0.log" Sep 08 07:41:50.706770 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:50.706734 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6cwdg_5ccc54f6-419a-4e85-be30-00100141ee31/manager/0.log" Sep 08 07:41:50.815199 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:50.815168 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79c9ffb96f-2bss8_8a2524d1-876e-4db1-ba27-84151cdfc850/copy-manifests/0.log" Sep 08 07:41:50.821353 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:50.821330 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79c9ffb96f-2bss8_8a2524d1-876e-4db1-ba27-84151cdfc850/manager/0.log" Sep 08 07:41:51.596564 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:51.596533 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29814210-5nx9f_e9f806e5-ecbb-433b-87fa-25b787949ac6/cleanup/0.log" Sep 08 07:41:53.439571 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:53.439541 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk_d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc/istio-proxy/0.log" Sep 08 07:41:53.550230 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:53.550188 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wh4dq_cdd1284b-171c-44ac-8344-fbdfe35898f9/discovery/0.log" Sep 08 07:41:53.878266 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:53.878233 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdwbm_6bb0f622-eca8-4bfd-9663-a720190f39c9/istio-proxy/1.log" Sep 08 07:41:53.985661 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:53.985628 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-wcf8b_8c23d559-3fc3-4c49-bd81-bcc28552a895/payload-pre-processing/0.log" Sep 08 07:41:54.101037 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:54.101007 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwl9h_9f88a82d-71c8-4d94-a9ff-e93e5b7f704c/payload-processing/0.log" Sep 08 07:41:54.746191 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:54.746162 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-bnbns_b94817e7-26f8-4f07-8bc8-8a0bdd3c1ce5/main/0.log" Sep 08 07:41:54.959267 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:41:54.959232 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnlckh_2bdae08d-581c-4435-8d39-d5b53ae69435/main/0.log" Sep 08 07:42:04.565706 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:04.565676 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gj26q_fe3c3143-6d3e-47ae-a9f5-c72cbcbb8a43/global-pull-secret-syncer/0.log" Sep 08 07:42:04.708645 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:04.708617 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ptdrw_0d3cb711-52d1-4457-a3e2-ce1efbefbfe1/konnectivity-agent/0.log" Sep 08 07:42:04.776428 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:04.776402 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-34.ec2.internal_6b5baafa68872d0e1a89c3b7fe397e3a/haproxy/0.log" Sep 08 07:42:10.764725 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:10.764655 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d255f57e-7e27-46b5-8900-15b3d01208ea/alertmanager/0.log" Sep 08 07:42:10.790354 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:10.790326 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d255f57e-7e27-46b5-8900-15b3d01208ea/config-reloader/0.log" Sep 08 07:42:10.813047 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:10.813022 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d255f57e-7e27-46b5-8900-15b3d01208ea/kube-rbac-proxy-web/0.log" Sep 08 07:42:10.837323 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:10.837299 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d255f57e-7e27-46b5-8900-15b3d01208ea/kube-rbac-proxy/0.log" Sep 08 07:42:10.863297 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:10.863270 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d255f57e-7e27-46b5-8900-15b3d01208ea/kube-rbac-proxy-metric/0.log" Sep 08 07:42:10.887025 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:10.887002 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d255f57e-7e27-46b5-8900-15b3d01208ea/prom-label-proxy/0.log" Sep 08 07:42:10.910148 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:10.910124 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d255f57e-7e27-46b5-8900-15b3d01208ea/init-config-reloader/0.log" Sep 08 07:42:10.979734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:10.979703 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-8lkjk_c1072ac4-aa72-492a-bf06-3a0f82c4f522/kube-state-metrics/0.log" Sep 08 07:42:11.001406 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.001379 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-8lkjk_c1072ac4-aa72-492a-bf06-3a0f82c4f522/kube-rbac-proxy-main/0.log" Sep 08 07:42:11.023339 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.023276 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-8lkjk_c1072ac4-aa72-492a-bf06-3a0f82c4f522/kube-rbac-proxy-self/0.log" Sep 08 07:42:11.080589 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.080562 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-4sgbs_3d9f7bc4-c7a2-4268-b7a1-dcd247b58efa/monitoring-plugin/0.log" Sep 08 07:42:11.202440 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.202410 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v7cql_71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec/node-exporter/0.log" Sep 08 07:42:11.223436 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.223391 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v7cql_71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec/kube-rbac-proxy/0.log" Sep 08 07:42:11.246875 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.246840 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v7cql_71e3d7cd-db7c-4757-9a8d-d42d4d7e2cec/init-textfile/0.log" Sep 08 07:42:11.352457 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.352382 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-c6l2h_f44eadf3-c51c-418e-b9ef-22266ddf6163/kube-rbac-proxy-main/0.log" Sep 08 07:42:11.378472 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.378446 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-c6l2h_f44eadf3-c51c-418e-b9ef-22266ddf6163/kube-rbac-proxy-self/0.log" Sep 08 07:42:11.399294 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.399268 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-c6l2h_f44eadf3-c51c-418e-b9ef-22266ddf6163/openshift-state-metrics/0.log" Sep 08 07:42:11.775939 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.775909 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7fc5f687c4-nbb2b_3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7/telemeter-client/0.log" Sep 08 07:42:11.802794 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.802762 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7fc5f687c4-nbb2b_3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7/reload/0.log" Sep 08 07:42:11.826734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:11.826651 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7fc5f687c4-nbb2b_3ec9ebfd-c95c-49d3-9df1-3f20ca8c2ad7/kube-rbac-proxy/0.log" Sep 08 07:42:14.762247 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.762191 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m"] Sep 08 07:42:14.762663 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.762649 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9a2ce0e-57ba-482d-a095-cfce4484e784" containerName="main" Sep 08 07:42:14.762726 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.762667 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9a2ce0e-57ba-482d-a095-cfce4484e784" containerName="main" Sep 08 07:42:14.762766 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.762751 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9a2ce0e-57ba-482d-a095-cfce4484e784" containerName="main" Sep 08 07:42:14.765918 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.765900 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.768493 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.768468 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jg2tn\"/\"default-dockercfg-jjph5\"" Sep 08 07:42:14.768626 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.768470 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jg2tn\"/\"openshift-service-ca.crt\"" Sep 08 07:42:14.768626 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.768529 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jg2tn\"/\"kube-root-ca.crt\"" Sep 08 07:42:14.774616 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.774593 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m"] Sep 08 07:42:14.778658 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.778628 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-proc\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.778772 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.778697 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-podres\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.778772 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.778735 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qllj9\" (UniqueName: \"kubernetes.io/projected/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-kube-api-access-qllj9\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.778772 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.778769 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-sys\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.778909 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.778851 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-lib-modules\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.879948 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.879900 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qllj9\" (UniqueName: \"kubernetes.io/projected/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-kube-api-access-qllj9\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.880167 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.879959 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-sys\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.880167 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.879989 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-lib-modules\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.880167 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.880036 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-proc\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.880167 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.880110 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-proc\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.880167 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.880122 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-podres\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.880435 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.880188 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-lib-modules\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.880435 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.880200 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-podres\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.880435 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.880109 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-sys\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:14.888557 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:14.888533 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qllj9\" (UniqueName: \"kubernetes.io/projected/279c844f-eeeb-41ab-b1ca-3b5d484ba3e6-kube-api-access-qllj9\") pod \"perf-node-gather-daemonset-zbv8m\" (UID: \"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6\") " pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:15.076794 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:15.076700 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:15.349340 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:15.349317 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m"] Sep 08 07:42:15.350726 ip-10-0-142-34 kubenswrapper[2571]: W0908 07:42:15.350695 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod279c844f_eeeb_41ab_b1ca_3b5d484ba3e6.slice/crio-52fd46ccabeeb024d4c063e04c93065203520c0fd5824cc1cdb1970df466bc92 WatchSource:0}: Error finding container 52fd46ccabeeb024d4c063e04c93065203520c0fd5824cc1cdb1970df466bc92: Status 404 returned error can't find the container with id 52fd46ccabeeb024d4c063e04c93065203520c0fd5824cc1cdb1970df466bc92 Sep 08 07:42:15.392280 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:15.392248 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" event={"ID":"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6","Type":"ContainerStarted","Data":"52fd46ccabeeb024d4c063e04c93065203520c0fd5824cc1cdb1970df466bc92"} Sep 08 07:42:16.399702 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:16.399661 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" event={"ID":"279c844f-eeeb-41ab-b1ca-3b5d484ba3e6","Type":"ContainerStarted","Data":"43bc27f74904dd89a94b4666701fd59934545dd64cef30059eee946e5dcb9ded"} Sep 08 07:42:16.400189 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:16.399726 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:16.418105 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:16.418050 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" podStartSLOduration=2.41803246 podStartE2EDuration="2.41803246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 07:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 07:42:16.415043714 +0000 UTC m=+2251.093006859" watchObservedRunningTime="2026-09-08 07:42:16.41803246 +0000 UTC m=+2251.095995576" Sep 08 07:42:16.635775 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:16.635746 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l246k_594b1074-cee3-42b8-9ec1-ded1a72984c4/dns/0.log" Sep 08 07:42:16.657017 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:16.656924 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l246k_594b1074-cee3-42b8-9ec1-ded1a72984c4/kube-rbac-proxy/0.log" Sep 08 07:42:16.793787 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:16.793758 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-988kt_4ace1cd1-322a-41ec-82df-53bc6914bd50/dns-node-resolver/0.log" Sep 08 07:42:17.358756 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:17.358725 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5bf64cffd6-qdfp2_733d6b54-2cf9-4d27-adb9-699a7e4a9e59/registry/0.log" Sep 08 07:42:17.378681 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:17.378654 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-drmx7_b05cd8e7-67da-40fe-a01d-d766c7d73ef9/node-ca/0.log" Sep 08 07:42:18.363664 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:18.363636 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk_d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc/istio-proxy/0.log" Sep 08 07:42:18.648010 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:18.647901 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wh4dq_cdd1284b-171c-44ac-8344-fbdfe35898f9/discovery/0.log" Sep 08 07:42:18.719895 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:18.719863 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdwbm_6bb0f622-eca8-4bfd-9663-a720190f39c9/istio-proxy/0.log" Sep 08 07:42:18.728749 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:18.728724 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdwbm_6bb0f622-eca8-4bfd-9663-a720190f39c9/istio-proxy/1.log" Sep 08 07:42:18.761136 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:18.761100 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-wcf8b_8c23d559-3fc3-4c49-bd81-bcc28552a895/payload-pre-processing/0.log" Sep 08 07:42:18.796081 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:18.796051 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwl9h_9f88a82d-71c8-4d94-a9ff-e93e5b7f704c/payload-processing/0.log" Sep 08 07:42:19.742501 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:19.742472 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rtxzw_e1925de9-7fbe-4cad-9181-eab4f6240e73/serve-healthcheck-canary/0.log" Sep 08 07:42:20.209508 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:20.209479 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9nwvl_c6f012a0-c7b8-4f72-93f4-c337275d2046/kube-rbac-proxy/0.log" Sep 08 07:42:20.234963 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:20.232979 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9nwvl_c6f012a0-c7b8-4f72-93f4-c337275d2046/exporter/0.log" Sep 08 07:42:20.259653 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:20.259627 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9nwvl_c6f012a0-c7b8-4f72-93f4-c337275d2046/extractor/0.log" Sep 08 07:42:22.414975 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:22.414950 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jg2tn/perf-node-gather-daemonset-zbv8m" Sep 08 07:42:22.757491 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:22.757465 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6qbws_d4b16240-e668-43ca-a967-2649668f1951/server/0.log" Sep 08 07:42:22.832627 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:22.832592 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6cwdg_5ccc54f6-419a-4e85-be30-00100141ee31/manager/0.log" Sep 08 07:42:22.855433 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:22.855412 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79c9ffb96f-2bss8_8a2524d1-876e-4db1-ba27-84151cdfc850/manager/0.log" Sep 08 07:42:22.877528 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:22.877503 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79c9ffb96f-2bss8_8a2524d1-876e-4db1-ba27-84151cdfc850/copy-manifests/0.log" Sep 08 07:42:24.435623 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:24.435589 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-767bfdf788-c2kfp_ed7c8ccd-be8f-4ac2-890d-d63fab8388bc/manager/0.log" Sep 08 07:42:25.557584 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:25.557550 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk_d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc/istio-proxy/0.log" Sep 08 07:42:25.833805 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:25.833718 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wh4dq_cdd1284b-171c-44ac-8344-fbdfe35898f9/discovery/0.log" Sep 08 07:42:25.882163 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:25.882128 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdwbm_6bb0f622-eca8-4bfd-9663-a720190f39c9/istio-proxy/0.log" Sep 08 07:42:25.890731 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:25.890707 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdwbm_6bb0f622-eca8-4bfd-9663-a720190f39c9/istio-proxy/1.log" Sep 08 07:42:25.918050 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:25.918022 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-wcf8b_8c23d559-3fc3-4c49-bd81-bcc28552a895/payload-pre-processing/0.log" Sep 08 07:42:25.943390 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:25.943358 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwl9h_9f88a82d-71c8-4d94-a9ff-e93e5b7f704c/payload-processing/0.log" Sep 08 07:42:27.498920 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:27.498888 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hvprk_d2e92058-a9fc-4f2c-a52e-13abb9d5ebcc/istio-proxy/0.log" Sep 08 07:42:27.780734 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:27.780661 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wh4dq_cdd1284b-171c-44ac-8344-fbdfe35898f9/discovery/0.log" Sep 08 07:42:27.847658 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:27.847628 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdwbm_6bb0f622-eca8-4bfd-9663-a720190f39c9/istio-proxy/0.log" Sep 08 07:42:27.856410 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:27.856384 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdwbm_6bb0f622-eca8-4bfd-9663-a720190f39c9/istio-proxy/1.log" Sep 08 07:42:27.882173 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:27.882145 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-wcf8b_8c23d559-3fc3-4c49-bd81-bcc28552a895/payload-pre-processing/0.log" Sep 08 07:42:27.908622 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:27.908589 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wwl9h_9f88a82d-71c8-4d94-a9ff-e93e5b7f704c/payload-processing/0.log" Sep 08 07:42:31.295023 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:31.294991 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-678bx_18c085d6-4085-42a9-bf4c-ad13ad01f7e6/kube-multus/0.log" Sep 08 07:42:31.486867 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:31.486843 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dtfw9_73d46ded-8a5c-46ba-abda-1359a34cd12c/kube-multus-additional-cni-plugins/0.log" Sep 08 07:42:31.507983 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:31.507957 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dtfw9_73d46ded-8a5c-46ba-abda-1359a34cd12c/egress-router-binary-copy/0.log" Sep 08 07:42:31.529860 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:31.529836 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dtfw9_73d46ded-8a5c-46ba-abda-1359a34cd12c/cni-plugins/0.log" Sep 08 07:42:31.550635 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:31.550566 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dtfw9_73d46ded-8a5c-46ba-abda-1359a34cd12c/bond-cni-plugin/0.log" Sep 08 07:42:31.574323 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:31.574298 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dtfw9_73d46ded-8a5c-46ba-abda-1359a34cd12c/routeoverride-cni/0.log" Sep 08 07:42:31.596305 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:31.596274 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dtfw9_73d46ded-8a5c-46ba-abda-1359a34cd12c/whereabouts-cni-bincopy/0.log" Sep 08 07:42:31.617182 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:31.617154 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dtfw9_73d46ded-8a5c-46ba-abda-1359a34cd12c/whereabouts-cni/0.log" Sep 08 07:42:31.882549 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:31.882465 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8k4kq_b282970a-79b3-42c5-8200-192bf1b0ac43/network-metrics-daemon/0.log" Sep 08 07:42:31.903134 ip-10-0-142-34 kubenswrapper[2571]: I0908 07:42:31.903106 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8k4kq_b282970a-79b3-42c5-8200-192bf1b0ac43/kube-rbac-proxy/0.log"