Sep 05 07:46:37.055037 ip-10-0-140-64 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 05 07:46:37.055053 ip-10-0-140-64 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 05 07:46:37.055062 ip-10-0-140-64 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 05 07:46:37.055414 ip-10-0-140-64 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 05 07:46:47.231120 ip-10-0-140-64 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 05 07:46:47.231137 ip-10-0-140-64 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 42eec94445ad49119f3621bb01b4e0f8 --
Sep 05 07:48:37.600305 ip-10-0-140-64 systemd[1]: Starting Kubernetes Kubelet...
Sep 05 07:48:38.030795 ip-10-0-140-64 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 07:48:38.030795 ip-10-0-140-64 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 05 07:48:38.030795 ip-10-0-140-64 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 07:48:38.030795 ip-10-0-140-64 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 05 07:48:38.030795 ip-10-0-140-64 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 07:48:38.032635 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.032553    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 05 07:48:38.040648 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040623    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:38.040648 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040643    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:38.040648 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040647    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:38.040648 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040650    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:38.040648 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040652    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:38.040648 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040655    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040659    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040662    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040665    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040667    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040670    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040673    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040675    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040680    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040685    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040688    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040691    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040694    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040697    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040700    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040702    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040705    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040708    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040710    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040713    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:38.040859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040716    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040720    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040724    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040727    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040730    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040733    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040736    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040739    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040742    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040745    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040747    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040750    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040753    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040755    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040758    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040761    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040763    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040766    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040768    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:38.041357 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040771    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040774    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040776    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040779    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040781    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040784    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040788    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040790    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040793    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040796    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040798    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040802    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040805    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040808    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040811    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040814    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040817    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040819    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040822    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:38.041825 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040825    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040828    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040830    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040833    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040835    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040838    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040841    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040843    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040846    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040848    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040851    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040854    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040857    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040859    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040862    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040865    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040869    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040871    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040874    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040877    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:38.042297 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040880    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040883    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040886    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040888    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040891    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040895    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040898    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040900    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040903    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040906    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040908    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040912    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040914    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040917    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040919    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.040922    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041710    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041718    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041721    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:38.042812 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041724    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041727    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041730    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041733    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041737    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041740    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041742    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041745    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041748    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041750    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041752    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041755    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041758    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041760    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041764    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041766    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041769    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041772    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041776    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:38.043276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041780    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041783    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041786    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041789    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041792    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041794    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041798    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041802    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041804    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041807    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041811    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041814    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041817    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041820    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041823    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041825    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041828    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041831    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041834    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041836    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:38.043741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041839    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041842    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041844    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041847    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041849    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041852    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041854    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041857    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041860    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041862    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041865    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041868    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041871    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041874    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041877    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041879    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041882    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041884    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041887    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041889    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:38.044255 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041892    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041894    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041896    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041899    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041903    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041907    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041910    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041912    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041915    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041917    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041920    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041922    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041925    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041927    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041930    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041933    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041935    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041938    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041940    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041943    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:38.044745 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041945    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041948    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041951    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041953    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041956    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041959    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041962    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041965    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041967    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041970    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041972    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041975    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041978    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041980    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041983    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041985    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.041988    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042395    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042400    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042403    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:38.045237 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042405    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042409    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042412    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042414    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042417    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042420    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042422    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042425    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042427    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042430    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042433    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042435    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042438    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042441    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042444    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042447    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042449    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042453    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042456    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042458    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:38.045719 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042461    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042463    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042466    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042468    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042471    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042473    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042476    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042478    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042481    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042484    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042486    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042490    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042494    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042496    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042499    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042502    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042504    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042507    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042510    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042513    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:38.046215 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042515    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042517    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042520    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042523    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042525    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042528    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042535    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042537    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042540    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042544    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042546    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042549    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042551    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042554    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042556    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042559    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042561    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042564    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042566    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042569    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:38.046701 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042572    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042574    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042576    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042579    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042581    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042584    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042586    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042590    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042592    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042595    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042599    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042602    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042605    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042608    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042611    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042614    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042618    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042621    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042625    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:38.047230 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042628    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042631    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042636    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042638    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042641    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042643    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042646    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042648    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042651    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042654    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042656    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042658    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042661    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042664    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042666    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042678    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.042681    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042753    2569 flags.go:64] FLAG: --address="0.0.0.0"
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042789    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042795    2569 flags.go:64] FLAG: --anonymous-auth="true"
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042800    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 05 07:48:38.047679 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042804    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042807    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042811    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042816    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042819    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042822    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042825    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042829    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042832    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042835    2569 flags.go:64] FLAG: --cgroup-root=""
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042839    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042842    2569 flags.go:64] FLAG: --client-ca-file=""
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042845    2569 flags.go:64] FLAG: --cloud-provider="external"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042850    2569 flags.go:64] FLAG: --cluster-dns="[]"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042855    2569 flags.go:64] FLAG: --cluster-domain=""
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042858    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042861    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042864    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042867    2569 flags.go:64] FLAG: --container-log-max-files="5"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042871    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042874    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042877    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042880    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042883    2569 flags.go:64] FLAG: --contention-profiling="false"
Sep 05 07:48:38.048185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042886    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042889    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042892    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042895    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042898    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042901    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042904    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042907    2569 flags.go:64] FLAG: --enable-load-reader="false"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042909    2569 flags.go:64] FLAG: --enable-server="true"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042912    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042916    2569 flags.go:64] FLAG: --event-burst="100"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042920    2569 flags.go:64] FLAG: --event-qps="50"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042922    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042925    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042929    2569 flags.go:64] FLAG: --eviction-hard=""
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042932    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042935    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042938    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042941    2569 flags.go:64] FLAG: --eviction-soft=""
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042944    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042947    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042951    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042954    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042957    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042960    2569 flags.go:64] FLAG: --fail-swap-on="true"
Sep 05 07:48:38.048751 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042963    2569 flags.go:64] FLAG: --feature-gates=""
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042967    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042970    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042973    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042976    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042979    2569 flags.go:64] FLAG: --healthz-port="10248"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042982    2569 flags.go:64] FLAG: --help="false"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042984    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042987    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042990    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042993    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.042996    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043000    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043003    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043005    2569 flags.go:64] FLAG: --image-service-endpoint=""
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043008    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043011    2569 flags.go:64] FLAG: --kube-api-burst="100"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043014    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043017    2569 flags.go:64] FLAG: --kube-api-qps="50"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043020    2569 flags.go:64] FLAG: --kube-reserved=""
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043023    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043025    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043028    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043031    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 05 07:48:38.049537 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043034    2569 flags.go:64] FLAG: --lock-file=""
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043036    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043039    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043056    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043063    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043075    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043078    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043081    2569 flags.go:64] FLAG: --logging-format="text"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043084    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043087    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043090    2569 flags.go:64] FLAG: --manifest-url=""
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043093    2569 flags.go:64] FLAG: --manifest-url-header=""
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043102    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043105    2569 flags.go:64] FLAG: --max-open-files="1000000"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043109    2569 flags.go:64] FLAG: --max-pods="110"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043112    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043115    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043118    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043121    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043124    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043126    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043129    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043136    2569 flags.go:64] FLAG: --node-status-max-images="50"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043139    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043142    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 05 07:48:38.050131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043145    2569 flags.go:64] FLAG: --pod-cidr=""
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043148    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043154    2569 flags.go:64] FLAG: --pod-manifest-path=""
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043157    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043160    2569 flags.go:64] FLAG: --pods-per-core="0"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043163    2569 flags.go:64] FLAG: --port="10250"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043166    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043169    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04b7227f5166aee1a"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043172    2569 flags.go:64] FLAG: --qos-reserved=""
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043175    2569 flags.go:64] FLAG: --read-only-port="10255"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043177    2569 flags.go:64] FLAG: --register-node="true"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043182    2569 flags.go:64] FLAG: --register-with-taints=""
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043185    2569 flags.go:64] FLAG: --registry-burst="10"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043189    2569 flags.go:64] FLAG: --registry-qps="5"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043191    2569 flags.go:64] FLAG: --reserved-cpus=""
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043194    2569 flags.go:64] FLAG: --reserved-memory=""
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043198    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043201    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043204    2569 flags.go:64] FLAG: --rotate-certificates="false"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043207    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043209    2569 flags.go:64] FLAG: --runonce="false"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043212    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043215    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043218    2569 flags.go:64] FLAG: --seccomp-default="false"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043220    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 05 07:48:38.050779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043223    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043228    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043231    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043234    2569 flags.go:64] FLAG: --storage-driver-password="root"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043237    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043240    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043242    2569 flags.go:64] FLAG: --storage-driver-user="root"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043245    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043248    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043251    2569 flags.go:64] FLAG: --system-cgroups=""
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043254    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043259    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043262    2569 flags.go:64] FLAG: --tls-cert-file=""
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043264    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043269    2569 flags.go:64] FLAG: --tls-min-version=""
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043272    2569 flags.go:64] FLAG: --tls-private-key-file=""
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043274    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043277    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043281    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043284    2569 flags.go:64] FLAG: --v="2"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043288    2569 flags.go:64] FLAG: --version="false"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043293    2569 flags.go:64] FLAG: --vmodule=""
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043296    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043300    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043433    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:38.051403 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043437    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043439    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043442    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043445    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043448    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043450    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043453    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043455    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043459    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043462    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043464    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043467    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043469    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043472    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043474    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043477    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043479    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043482    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043485    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043487    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:38.052009 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043490    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043492    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043495    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043497    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043500    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043503    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043505    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043508    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043510    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043513    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043516    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043518    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043521    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043523    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043526    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043528    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043531    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043533    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043537    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043540    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:38.052502 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043544    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043547    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043550    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043552    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043555    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043557    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043560    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043562    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043565    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043567    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043570    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043572    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043575    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043577    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043580    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043582    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043585    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043588    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043591    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:38.053003 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043595    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043598    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043601    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043604    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043607    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043610    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043612    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043614    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043617    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043619    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043622    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043624    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043627    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043630    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043633    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043635    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043638    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043641    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043643    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043646    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:38.053498 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043648    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043651    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043653    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043656    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043658    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043661    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043664    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043666    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043669    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043672    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043674    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043677    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043679    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043682    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043684    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043688    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043692    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043694    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:38.053988 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.043697    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.043703    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.051691    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.051705    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.051719    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.051725    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051788    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051793    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051796    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051801    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051805    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051809    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051812    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051816    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:38.054477 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051819    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051822    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051824    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051827    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051830    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051833    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051836    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051839    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051842    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051845    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051848    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051851    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051853    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051856    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051858    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051861    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051863    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051866    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051868    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051871    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:38.054854 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051873    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051880    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051883    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051886    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051888    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051891    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051894    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051896    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051899    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051901    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051904    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051906    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051910    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051913    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051916    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051918    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051921    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051923    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051926    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:38.055354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051929    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051931    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051934    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051936    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051939    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051942    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051944    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051947    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051950    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051952    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051954    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051957    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051959    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051961    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051964    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051968    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051971    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051973    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051975    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051978    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:38.055797 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051981    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051983    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051986    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051988    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051990    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051993    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051995    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.051998    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052000    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052002    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052005    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052007    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052010    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052013    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052015    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052018    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052020    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052023    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052025    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052028    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:38.056288 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052030    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052033    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052035    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052037    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052056    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052060    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052064    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052066    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052070    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052073    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052076    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052078    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.052083    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052216    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052221    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 07:48:38.056762 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052224    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052227    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052230    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052233    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052236    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052239    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052241    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052244    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052246    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052248    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052251    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052253    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052256    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052258    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052261    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052264    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052266    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052269    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052271    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052274    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 07:48:38.057132 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052276    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052278    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052281    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052283    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052286    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052289    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052292    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052295    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052297    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052299    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052302    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052304    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052307    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052310    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052314    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052317    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052320    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052323    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052325    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 07:48:38.057602 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052328    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052330    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052333    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052336    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052338    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052341    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052343    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052346    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052348    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052351    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052353    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052356    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052358    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052362    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052365    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052368    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052371    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052374    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052377    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052379    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 07:48:38.058076 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052382    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052385    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052387    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052390    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052393    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052395    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052398    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052400    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052402    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052405    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052407    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052410    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052412    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052415    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052417    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052419    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052422    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052424    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052427    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052430    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 07:48:38.058561 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052432    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052435    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052437    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052440    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052442    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052444    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052447    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052449    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052454    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052458    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052462    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052465    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052467    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052471    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052473    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052476    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052479    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 07:48:38.059058 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:38.052481    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 07:48:38.059455 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.052486    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 07:48:38.059455 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.052592    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 05 07:48:38.059455 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.055063    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 05 07:48:38.059455 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.056144    2569 server.go:1019] "Starting client certificate rotation"
Sep 05 07:48:38.059455 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.056247    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 05 07:48:38.059455 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.056278    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 05 07:48:38.059455 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.058972    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 05 07:48:38.060176 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.060158    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 05 07:48:38.072551 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.072533    2569 log.go:25] "Validated CRI v1 runtime API"
Sep 05 07:48:38.077781 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.077764    2569 log.go:25] "Validated CRI v1 image API"
Sep 05 07:48:38.078694 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.078677    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 05 07:48:38.083891 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.083873    2569 fs.go:135] Filesystem UUIDs: map[1b0bc116-d3f5-486b-beaa-5bd3af794bdf:/dev/nvme0n1p3 6b876deb-6ee7-4fb1-aeef-76a0a7de3f67:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 05 07:48:38.083964 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.083891    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 05 07:48:38.090414 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.090306    2569 manager.go:216] Machine: {Timestamp:2026-09-05 07:48:38.088787408 +0000 UTC m=+0.375590894 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3082056 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ac2a211a30d4e717fe7ecbbcd592f SystemUUID:ec2ac2a2-11a3-0d4e-717f-e7ecbbcd592f BootID:42eec944-45ad-4911-9f36-21bb01b4e0f8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1f:c1:60:d9:d7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1f:c1:60:d9:d7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:c5:f5:a0:66:3c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 05 07:48:38.090414 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.090409    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 05 07:48:38.090532 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.090483    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 05 07:48:38.091082 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.091063    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 05 07:48:38.094106 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.094085    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 05 07:48:38.094244 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.094111    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-64.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 05 07:48:38.094989 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.094978    2569 topology_manager.go:138] "Creating topology manager with none policy"
Sep 05 07:48:38.095020 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.094992    2569 container_manager_linux.go:309] "Creating device plugin manager"
Sep 05 07:48:38.095020 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.095004    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 05 07:48:38.095020 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.095018    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 05 07:48:38.096932 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.096922    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 05 07:48:38.101483 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.101472    2569 state_mem.go:36] "Initialized new in-memory state store"
Sep 05 07:48:38.101595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.101585    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 05 07:48:38.101626 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.101617    2569 kubelet.go:486] "Attempting to sync node with API server"
Sep 05 07:48:38.101652 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.101629    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 05 07:48:38.101652 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.101644    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 05 07:48:38.101652 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.101653    2569 kubelet.go:398] "Adding apiserver pod source"
Sep 05 07:48:38.101738 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.101661    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 05 07:48:38.102231 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.102184    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 05 07:48:38.102286 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.102243    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 05 07:48:38.104733 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.104718    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 05 07:48:38.107932 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.107919    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 05 07:48:38.107978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.107948    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 05 07:48:38.108164 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108155    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 05 07:48:38.108207 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108169    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 05 07:48:38.108207 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108175    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 05 07:48:38.108207 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108180    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 05 07:48:38.108207 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108187    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 05 07:48:38.108207 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108202    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 05 07:48:38.108754 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108745    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 05 07:48:38.108785 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108755    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 05 07:48:38.108785 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108770    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 05 07:48:38.108785 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108775    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 05 07:48:38.108864 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108791    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 05 07:48:38.108864 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.108799    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 05 07:48:38.109754 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.109743    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 05 07:48:38.109787 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.109756    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 05 07:48:38.112559 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.112494    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-64.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 05 07:48:38.114213 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.113607    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 05 07:48:38.115547 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.115529    2569 server.go:1268] "Started kubelet"
Sep 05 07:48:38.115621 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.115590    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 05 07:48:38.115677 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.115654    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 05 07:48:38.115719 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.115701    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 05 07:48:38.116247 ip-10-0-140-64 systemd[1]: Started Kubernetes Kubelet.
Sep 05 07:48:38.118492 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.118477    2569 server.go:310] "Adding debug handlers to kubelet server"
Sep 05 07:48:38.119708 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.119690    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 05 07:48:38.121589 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.121571    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pr2t6"
Sep 05 07:48:38.122234 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.122216    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 05 07:48:38.123444 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.123428    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-64.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 05 07:48:38.124646 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.123464    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-64.ec2.internal.18d25d3ce37c41ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-64.ec2.internal,UID:ip-10-0-140-64.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-64.ec2.internal,},FirstTimestamp:2026-09-05 07:48:38.115500494 +0000 UTC m=+0.402303979,LastTimestamp:2026-09-05 07:48:38.115500494 +0000 UTC m=+0.402303979,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-64.ec2.internal,}"
Sep 05 07:48:38.125129 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.125112    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 05 07:48:38.125221 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.125130    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 05 07:48:38.125382 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.125319    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:38.125911 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.125886    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 05 07:48:38.126032 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.125923    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 05 07:48:38.126149 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.126139    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 05 07:48:38.126262 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.126244    2569 factory.go:55] Registering systemd factory
Sep 05 07:48:38.126365 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.126265    2569 factory.go:223] Registration of the systemd container factory successfully
Sep 05 07:48:38.126365 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.126298    2569 reconstruct.go:97] "Volume reconstruction finished"
Sep 05 07:48:38.126365 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.126309    2569 reconciler.go:29] "Reconciler: start to sync state"
Sep 05 07:48:38.126509 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.126500    2569 factory.go:153] Registering CRI-O factory
Sep 05 07:48:38.126509 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.126510    2569 factory.go:223] Registration of the crio container factory successfully
Sep 05 07:48:38.126593 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.126550    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 05 07:48:38.126593 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.126564    2569 factory.go:103] Registering Raw factory
Sep 05 07:48:38.126593 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.126574    2569 manager.go:1196] Started watching for new ooms in manager
Sep 05 07:48:38.127037 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.127015    2569 manager.go:349] Starting recovery of all containers
Sep 05 07:48:38.128341 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.128311    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 05 07:48:38.128458 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.128436    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-64.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 05 07:48:38.129492 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.129472    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pr2t6"
Sep 05 07:48:38.138683 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.138640    2569 manager.go:354] Recovery completed
Sep 05 07:48:38.143406 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.143394    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:38.145016 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.145001    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:38.145083 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.145028    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:38.145083 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.145067    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:38.145586 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.145572    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 05 07:48:38.145628 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.145586    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 05 07:48:38.145628 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.145600    2569 state_mem.go:36] "Initialized new in-memory state store"
Sep 05 07:48:38.145628 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.145578    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:38.146250 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.146237    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:38.146294 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.146265    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:38.146294 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.146284    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:38.147543 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.147532    2569 policy_none.go:49] "None policy: Start"
Sep 05 07:48:38.147581 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.147547    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 05 07:48:38.147581 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.147557    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 05 07:48:38.149516 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.149504    2569 policy_none.go:47] "Start"
Sep 05 07:48:38.181585 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.181571    2569 manager.go:337] "Starting Device Plugin manager"
Sep 05 07:48:38.205485 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.181604    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 05 07:48:38.205485 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.181614    2569 server.go:87] "Starting device plugin registration server"
Sep 05 07:48:38.205485 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.181800    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 05 07:48:38.205485 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.181813    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 05 07:48:38.205485 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.181890    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 05 07:48:38.205485 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.182176    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 05 07:48:38.205485 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.182186    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 05 07:48:38.205485 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.182451    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 05 07:48:38.205485 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.182509    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:38.220898 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.220878    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 05 07:48:38.222150 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.222134    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 05 07:48:38.222254 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.222178    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 05 07:48:38.222254 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.222197    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 05 07:48:38.222254 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.222231    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 05 07:48:38.224438 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.224415    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 07:48:38.282346 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.282303    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:38.283025 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.283008    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:38.283123 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.283037    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:38.283123 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.283079    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:38.283123 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.283104    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.290832 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.290819    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.290877 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.290838    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-64.ec2.internal\": node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:38.304753 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.304733    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:38.323110 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.323087    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal"]
Sep 05 07:48:38.323176 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.323159    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:38.324432 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.324419    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:38.324493 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.324443    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:38.324493 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.324462    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:38.326690 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.326678    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:38.326831 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.326818    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.326869 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.326847    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:38.327309 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.327293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df5961174929c21de5075c8cdb7e20e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal\" (UID: \"df5961174929c21de5075c8cdb7e20e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.327369 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.327333    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df5961174929c21de5075c8cdb7e20e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal\" (UID: \"df5961174929c21de5075c8cdb7e20e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.327680 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.327663    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:38.327732 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.327692    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:38.327732 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.327703    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:38.328289 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.328275    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:38.328350 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.328300    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:38.328350 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.328311    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:38.329978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.329962    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.330078 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.329992    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 07:48:38.330717 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.330700    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 07:48:38.330781 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.330731    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 07:48:38.330781 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.330748    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeHasSufficientPID"
Sep 05 07:48:38.356619 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.356599    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-64.ec2.internal\" not found" node="ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.360712 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.360698    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-64.ec2.internal\" not found" node="ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.404936 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.404914    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:38.428280 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.428258    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1120dc1be11d71eb0105a790715e7795-config\") pod \"kube-apiserver-proxy-ip-10-0-140-64.ec2.internal\" (UID: \"1120dc1be11d71eb0105a790715e7795\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.428353 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.428305    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df5961174929c21de5075c8cdb7e20e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal\" (UID: \"df5961174929c21de5075c8cdb7e20e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.428353 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.428322    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df5961174929c21de5075c8cdb7e20e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal\" (UID: \"df5961174929c21de5075c8cdb7e20e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.428353 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.428349    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df5961174929c21de5075c8cdb7e20e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal\" (UID: \"df5961174929c21de5075c8cdb7e20e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.428447 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.428359    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df5961174929c21de5075c8cdb7e20e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal\" (UID: \"df5961174929c21de5075c8cdb7e20e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.506020 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.506000    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:38.528683 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.528664    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1120dc1be11d71eb0105a790715e7795-config\") pod \"kube-apiserver-proxy-ip-10-0-140-64.ec2.internal\" (UID: \"1120dc1be11d71eb0105a790715e7795\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.528745 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.528695    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1120dc1be11d71eb0105a790715e7795-config\") pod \"kube-apiserver-proxy-ip-10-0-140-64.ec2.internal\" (UID: \"1120dc1be11d71eb0105a790715e7795\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.607153 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.607090    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:38.659547 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.659520    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.663013 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:38.662997    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:38.707426 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.707404    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:38.807958 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.807935    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:38.908464 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:38.908393    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:39.008930 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:39.008905    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:39.056405 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.056382    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 05 07:48:39.056756 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.056540    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 05 07:48:39.109831 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:39.109802    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-64.ec2.internal\" not found"
Sep 05 07:48:39.125371 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.125350    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 07:48:39.132540 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.132511    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-04 07:43:39 +0000 UTC" deadline="2028-06-09 08:46:58.761251838 +0000 UTC"
Sep 05 07:48:39.132540 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.132539    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15432h58m19.628716792s"
Sep 05 07:48:39.184471 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:39.184435    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1120dc1be11d71eb0105a790715e7795.slice/crio-d0334284d344d27494ecade627c244e4a11781733cd039b8abd5d94c1c29ed74 WatchSource:0}: Error finding container d0334284d344d27494ecade627c244e4a11781733cd039b8abd5d94c1c29ed74: Status 404 returned error can't find the container with id d0334284d344d27494ecade627c244e4a11781733cd039b8abd5d94c1c29ed74
Sep 05 07:48:39.184741 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:39.184723    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf5961174929c21de5075c8cdb7e20e2.slice/crio-87b1ee27af2e77090edee094df1867f290bbdc6faf0c8f8a5d2e08a70af498cb WatchSource:0}: Error finding container 87b1ee27af2e77090edee094df1867f290bbdc6faf0c8f8a5d2e08a70af498cb: Status 404 returned error can't find the container with id 87b1ee27af2e77090edee094df1867f290bbdc6faf0c8f8a5d2e08a70af498cb
Sep 05 07:48:39.189184 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.189163    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 07:48:39.225348 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.225324    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:39.225609 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.225571    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal" event={"ID":"1120dc1be11d71eb0105a790715e7795","Type":"ContainerStarted","Data":"d0334284d344d27494ecade627c244e4a11781733cd039b8abd5d94c1c29ed74"}
Sep 05 07:48:39.226654 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.226636    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal" event={"ID":"df5961174929c21de5075c8cdb7e20e2","Type":"ContainerStarted","Data":"87b1ee27af2e77090edee094df1867f290bbdc6faf0c8f8a5d2e08a70af498cb"}
Sep 05 07:48:39.236885 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.236868    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 05 07:48:39.238806 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.238794    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal"
Sep 05 07:48:39.247784 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:39.247772    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 05 07:48:40.850644 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:40.850579    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 07:48:41.037090 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.037064    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 07:48:41.368362 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.368331    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 07:48:41.444634 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.444600    2569 apiserver.go:52] "Watching apiserver"
Sep 05 07:48:41.454846 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.454820    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 05 07:48:41.455209 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.455183    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-bcxfm","openshift-network-diagnostics/network-check-target-7njm4","openshift-network-operator/iptables-alerter-shk85","kube-system/konnectivity-agent-v2j86","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5","openshift-cluster-node-tuning-operator/tuned-pff8t","openshift-dns/node-resolver-q754g","openshift-image-registry/node-ca-mgwk5","openshift-multus/multus-9tqz9","openshift-multus/multus-additional-cni-plugins-mvv55","openshift-ovn-kubernetes/ovnkube-node-xp7vx","kube-system/global-pull-secret-syncer-6nw26","kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal"]
Sep 05 07:48:41.457753 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.457717    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:41.457861 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.457810    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:48:41.459872 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.459851    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:41.459969 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.459929    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:48:41.462068 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.462040    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.464383 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.464364    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:48:41.465020 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.465000    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 05 07:48:41.465238 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.465210    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 05 07:48:41.465297 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.465236    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 05 07:48:41.465297 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.465226    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fc4kx\""
Sep 05 07:48:41.466406 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.466378    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.466852 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.466834    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 05 07:48:41.467020 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.466999    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 05 07:48:41.467145 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.467132    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 05 07:48:41.467367 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.467349    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-crlpv\""
Sep 05 07:48:41.468984 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.468968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.469307 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.469286    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 05 07:48:41.469414 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.469332    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 05 07:48:41.469414 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.469392    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s45kq\""
Sep 05 07:48:41.469547 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.469460    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 05 07:48:41.471237 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.471216    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.472033 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.471997    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-l7hb5\""
Sep 05 07:48:41.472174 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.472073    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 05 07:48:41.472310 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.472290    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 05 07:48:41.473586 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.473566    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 05 07:48:41.473655 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.473602    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.473831 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.473818    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7mc4j\""
Sep 05 07:48:41.473899 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.473883    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 05 07:48:41.476365 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.475801    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.476447 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.476400    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 05 07:48:41.476644 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.476624    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 05 07:48:41.477010 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.476928    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 05 07:48:41.477143 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.477127    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vrbgv\""
Sep 05 07:48:41.478348 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478328    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-modprobe-d\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.478430 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478362    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-systemd\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.478430 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-var-lib-kubelet\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.478430 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478410    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-lib-modules\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.478585 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478432    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-host\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.478585 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478456    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5387bb72-57d6-48da-8f99-365c22edc4dd-iptables-alerter-script\") pod \"iptables-alerter-shk85\" (UID: \"5387bb72-57d6-48da-8f99-365c22edc4dd\") " pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.478585 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478480    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-sys-fs\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.478585 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478499    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-sys\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.478585 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478541    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94dwb\" (UniqueName: \"kubernetes.io/projected/4dfe1004-84bf-4d78-b813-e966d95dbd4c-kube-api-access-94dwb\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.478585 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478567    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z7t5\" (UniqueName: \"kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5\") pod \"network-check-target-7njm4\" (UID: \"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc\") " pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478590    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-sysconfig\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-sysctl-conf\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478616    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478634    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478657    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-socket-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478680    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2fwk\" (UniqueName: \"kubernetes.io/projected/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-kube-api-access-j2fwk\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478707    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a07aa982-f058-45d8-b69d-f6f248285657-agent-certs\") pod \"konnectivity-agent-v2j86\" (UID: \"a07aa982-f058-45d8-b69d-f6f248285657\") " pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478728    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-tuned\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478752    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5387bb72-57d6-48da-8f99-365c22edc4dd-host-slash\") pod \"iptables-alerter-shk85\" (UID: \"5387bb72-57d6-48da-8f99-365c22edc4dd\") " pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478774    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6v8f\" (UniqueName: \"kubernetes.io/projected/5387bb72-57d6-48da-8f99-365c22edc4dd-kube-api-access-g6v8f\") pod \"iptables-alerter-shk85\" (UID: \"5387bb72-57d6-48da-8f99-365c22edc4dd\") " pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478796    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.478841 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-registration-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.479253 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-device-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.479253 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478889    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a07aa982-f058-45d8-b69d-f6f248285657-konnectivity-ca\") pod \"konnectivity-agent-v2j86\" (UID: \"a07aa982-f058-45d8-b69d-f6f248285657\") " pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:48:41.479253 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478911    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-etc-selinux\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.479253 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478932    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-sysctl-d\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.479253 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478954    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-run\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.479253 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478976    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-kubernetes\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.479253 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.478999    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2vk8\" (UniqueName: \"kubernetes.io/projected/6264509b-4382-45b2-a6f0-d8af26dd57f1-kube-api-access-d2vk8\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:41.479253 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.479031    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dfe1004-84bf-4d78-b813-e966d95dbd4c-tmp\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.479711 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.479478    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 05 07:48:41.479711 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.479513    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rtrjr\""
Sep 05 07:48:41.479711 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.479562    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 05 07:48:41.479711 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.479633    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 05 07:48:41.479711 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.479516    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 05 07:48:41.480493 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.480473    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.481379 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.481357    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 05 07:48:41.481450 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.481389    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 05 07:48:41.481865 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.481846    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-j5b74\""
Sep 05 07:48:41.482830 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.482809    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 05 07:48:41.484721 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.484704    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 05 07:48:41.484798 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.484740    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 05 07:48:41.484798 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.484756    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 05 07:48:41.484906 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.484817    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 05 07:48:41.484906 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.484879    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 05 07:48:41.485047 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.484919    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bxzsd\""
Sep 05 07:48:41.485099 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.485056    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 05 07:48:41.485209 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.485194    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:41.485278 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.485259    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:48:41.503070 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.502864    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5qk7r"
Sep 05 07:48:41.512132 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.512112    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5qk7r"
Sep 05 07:48:41.568251 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.568226    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 05 07:48:41.579473 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579450    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-run-ovn\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.579593 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579486    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-node-log\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.579593 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579512    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-sysconfig\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.579593 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579550    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-sysctl-conf\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.579593 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579565    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/632a9615-66ff-47b8-b9e5-4357b516b064-host\") pod \"node-ca-mgwk5\" (UID: \"632a9615-66ff-47b8-b9e5-4357b516b064\") " pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.579593 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579580    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-socket-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579596    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-run-netns\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579606    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-sysconfig\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-var-lib-cni-bin\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579655    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-cni-netd\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579679    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-env-overrides\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579713    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-sysctl-conf\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579733    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/97e3d2a4-be3f-487f-b2a3-380e78fed6e1-hosts-file\") pod \"node-resolver-q754g\" (UID: \"97e3d2a4-be3f-487f-b2a3-380e78fed6e1\") " pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579739    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-socket-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579759    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzzwj\" (UniqueName: \"kubernetes.io/projected/97e3d2a4-be3f-487f-b2a3-380e78fed6e1-kube-api-access-hzzwj\") pod \"node-resolver-q754g\" (UID: \"97e3d2a4-be3f-487f-b2a3-380e78fed6e1\") " pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579782    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-socket-dir-parent\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.579816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579815    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-var-lib-cni-multus\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6v8f\" (UniqueName: \"kubernetes.io/projected/5387bb72-57d6-48da-8f99-365c22edc4dd-kube-api-access-g6v8f\") pod \"iptables-alerter-shk85\" (UID: \"5387bb72-57d6-48da-8f99-365c22edc4dd\") " pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579863    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-device-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579878    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-system-cni-dir\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579896    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a07aa982-f058-45d8-b69d-f6f248285657-konnectivity-ca\") pod \"konnectivity-agent-v2j86\" (UID: \"a07aa982-f058-45d8-b69d-f6f248285657\") " pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-sysctl-d\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579946    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-device-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579946    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-run\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.579984    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8wr4\" (UniqueName: \"kubernetes.io/projected/632a9615-66ff-47b8-b9e5-4357b516b064-kube-api-access-q8wr4\") pod \"node-ca-mgwk5\" (UID: \"632a9615-66ff-47b8-b9e5-4357b516b064\") " pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-slash\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580024    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-ovnkube-config\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580073    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-run\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580088    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ljfm\" (UniqueName: \"kubernetes.io/projected/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-kube-api-access-7ljfm\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580126    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2vk8\" (UniqueName: \"kubernetes.io/projected/6264509b-4382-45b2-a6f0-d8af26dd57f1-kube-api-access-d2vk8\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580148    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6eeab356-b5db-4ccf-920e-3bc31edc9962-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580177    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-kubelet\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.580281 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580196    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-ovnkube-script-lib\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580202    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-sysctl-d\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580217    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-dbus\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580256    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580280    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-daemon-config\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580315    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-var-lib-openvswitch\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-log-socket\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580396    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-lib-modules\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580420    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-host\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a07aa982-f058-45d8-b69d-f6f248285657-konnectivity-ca\") pod \"konnectivity-agent-v2j86\" (UID: \"a07aa982-f058-45d8-b69d-f6f248285657\") " pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580459    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-host\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580457    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5387bb72-57d6-48da-8f99-365c22edc4dd-iptables-alerter-script\") pod \"iptables-alerter-shk85\" (UID: \"5387bb72-57d6-48da-8f99-365c22edc4dd\") " pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580558    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-lib-modules\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580541    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-sys-fs\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580588    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7t5\" (UniqueName: \"kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5\") pod \"network-check-target-7njm4\" (UID: \"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc\") " pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580593    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-sys-fs\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6eeab356-b5db-4ccf-920e-3bc31edc9962-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.580952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580633    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-ovn-node-metrics-cert\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580649    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580666    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-cni-dir\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580689    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6eeab356-b5db-4ccf-920e-3bc31edc9962-system-cni-dir\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580725    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6eeab356-b5db-4ccf-920e-3bc31edc9962-cnibin\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580796    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6eeab356-b5db-4ccf-920e-3bc31edc9962-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580820    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.580799    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580842    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2fwk\" (UniqueName: \"kubernetes.io/projected/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-kube-api-access-j2fwk\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580864    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-etc-kubernetes\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580911    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a07aa982-f058-45d8-b69d-f6f248285657-agent-certs\") pod \"konnectivity-agent-v2j86\" (UID: \"a07aa982-f058-45d8-b69d-f6f248285657\") " pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.580932    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs podName:6264509b-4382-45b2-a6f0-d8af26dd57f1 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:42.080907538 +0000 UTC m=+3.026229340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs") pod "network-metrics-daemon-bcxfm" (UID: "6264509b-4382-45b2-a6f0-d8af26dd57f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.580955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5387bb72-57d6-48da-8f99-365c22edc4dd-iptables-alerter-script\") pod \"iptables-alerter-shk85\" (UID: \"5387bb72-57d6-48da-8f99-365c22edc4dd\") " pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581031    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97e3d2a4-be3f-487f-b2a3-380e78fed6e1-tmp-dir\") pod \"node-resolver-q754g\" (UID: \"97e3d2a4-be3f-487f-b2a3-380e78fed6e1\") " pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581068    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-cnibin\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581095    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-tuned\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.581697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581119    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5387bb72-57d6-48da-8f99-365c22edc4dd-host-slash\") pod \"iptables-alerter-shk85\" (UID: \"5387bb72-57d6-48da-8f99-365c22edc4dd\") " pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581143    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581168    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-registration-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581191    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/632a9615-66ff-47b8-b9e5-4357b516b064-serviceca\") pod \"node-ca-mgwk5\" (UID: \"632a9615-66ff-47b8-b9e5-4357b516b064\") " pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581196    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5387bb72-57d6-48da-8f99-365c22edc4dd-host-slash\") pod \"iptables-alerter-shk85\" (UID: \"5387bb72-57d6-48da-8f99-365c22edc4dd\") " pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581209    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581233    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-run-systemd\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581260    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-registration-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581275    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-etc-openvswitch\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581308    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-run-openvswitch\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581350    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-etc-selinux\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581378    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-run-multus-certs\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581404    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6eeab356-b5db-4ccf-920e-3bc31edc9962-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581431    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhbdg\" (UniqueName: \"kubernetes.io/projected/6eeab356-b5db-4ccf-920e-3bc31edc9962-kube-api-access-zhbdg\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581455    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-systemd-units\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581476    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-etc-selinux\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.582595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-kubernetes\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581519    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-cni-bin\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581554    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-kubernetes\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581562    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-kubelet-config\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581586    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78shm\" (UniqueName: \"kubernetes.io/projected/4acfb359-d4f2-4e0a-a442-7871d3000d97-kube-api-access-78shm\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581639    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dfe1004-84bf-4d78-b813-e966d95dbd4c-tmp\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581686    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-modprobe-d\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581786    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-systemd\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581811    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-var-lib-kubelet\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581843    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-var-lib-kubelet\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581863    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-hostroot\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581883    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-run-k8s-cni-cncf-io\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581921    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-conf-dir\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581942    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-sys\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581948    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-modprobe-d\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581965    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94dwb\" (UniqueName: \"kubernetes.io/projected/4dfe1004-84bf-4d78-b813-e966d95dbd4c-kube-api-access-94dwb\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.581986    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4acfb359-d4f2-4e0a-a442-7871d3000d97-cni-binary-copy\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.583413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.582006    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-run-netns\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.584220 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.582079    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-var-lib-kubelet\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.584220 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.582081    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-systemd\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.584220 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.582137    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4dfe1004-84bf-4d78-b813-e966d95dbd4c-sys\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.584812 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.584792    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4dfe1004-84bf-4d78-b813-e966d95dbd4c-etc-tuned\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.584912 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.584846    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dfe1004-84bf-4d78-b813-e966d95dbd4c-tmp\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.585019 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.585004    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a07aa982-f058-45d8-b69d-f6f248285657-agent-certs\") pod \"konnectivity-agent-v2j86\" (UID: \"a07aa982-f058-45d8-b69d-f6f248285657\") " pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:48:41.586138 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.586117    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:41.586246 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.586143    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:41.586246 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.586157    2569 projected.go:196] Error preparing data for projected volume kube-api-access-9z7t5 for pod openshift-network-diagnostics/network-check-target-7njm4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:41.586246 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.586219    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5 podName:c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc nodeName:}" failed. No retries permitted until 2026-09-05 07:48:42.086201445 +0000 UTC m=+3.031523246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9z7t5" (UniqueName: "kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5") pod "network-check-target-7njm4" (UID: "c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:41.588219 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.588198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2vk8\" (UniqueName: \"kubernetes.io/projected/6264509b-4382-45b2-a6f0-d8af26dd57f1-kube-api-access-d2vk8\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:41.588459 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.588440    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6v8f\" (UniqueName: \"kubernetes.io/projected/5387bb72-57d6-48da-8f99-365c22edc4dd-kube-api-access-g6v8f\") pod \"iptables-alerter-shk85\" (UID: \"5387bb72-57d6-48da-8f99-365c22edc4dd\") " pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.591601 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.591575    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2fwk\" (UniqueName: \"kubernetes.io/projected/fe597340-d05d-40d4-90c8-7f4cb6fc6e01-kube-api-access-j2fwk\") pod \"aws-ebs-csi-driver-node-hf4l5\" (UID: \"fe597340-d05d-40d4-90c8-7f4cb6fc6e01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.593298 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.593280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94dwb\" (UniqueName: \"kubernetes.io/projected/4dfe1004-84bf-4d78-b813-e966d95dbd4c-kube-api-access-94dwb\") pod \"tuned-pff8t\" (UID: \"4dfe1004-84bf-4d78-b813-e966d95dbd4c\") " pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.682760 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682683    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-run-netns\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.682760 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682720    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-run-ovn\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.682760 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682742    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-node-log\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/632a9615-66ff-47b8-b9e5-4357b516b064-host\") pod \"node-ca-mgwk5\" (UID: \"632a9615-66ff-47b8-b9e5-4357b516b064\") " pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682788    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-run-netns\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682803    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-run-netns\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682815    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-run-ovn\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682817    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-node-log\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682842    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-var-lib-cni-bin\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682846    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-run-netns\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682863    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/632a9615-66ff-47b8-b9e5-4357b516b064-host\") pod \"node-ca-mgwk5\" (UID: \"632a9615-66ff-47b8-b9e5-4357b516b064\") " pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-var-lib-cni-bin\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-cni-netd\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682930    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-env-overrides\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/97e3d2a4-be3f-487f-b2a3-380e78fed6e1-hosts-file\") pod \"node-resolver-q754g\" (UID: \"97e3d2a4-be3f-487f-b2a3-380e78fed6e1\") " pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682963    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-cni-netd\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.682976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzzwj\" (UniqueName: \"kubernetes.io/projected/97e3d2a4-be3f-487f-b2a3-380e78fed6e1-kube-api-access-hzzwj\") pod \"node-resolver-q754g\" (UID: \"97e3d2a4-be3f-487f-b2a3-380e78fed6e1\") " pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.683045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-socket-dir-parent\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-socket-dir-parent\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/97e3d2a4-be3f-487f-b2a3-380e78fed6e1-hosts-file\") pod \"node-resolver-q754g\" (UID: \"97e3d2a4-be3f-487f-b2a3-380e78fed6e1\") " pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683100    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-var-lib-cni-multus\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683130    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-system-cni-dir\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683161    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8wr4\" (UniqueName: \"kubernetes.io/projected/632a9615-66ff-47b8-b9e5-4357b516b064-kube-api-access-q8wr4\") pod \"node-ca-mgwk5\" (UID: \"632a9615-66ff-47b8-b9e5-4357b516b064\") " pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683170    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-var-lib-cni-multus\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683186    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-slash\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683212    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683219    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-system-cni-dir\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683245    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-ovnkube-config\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683264    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683270    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ljfm\" (UniqueName: \"kubernetes.io/projected/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-kube-api-access-7ljfm\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683300    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6eeab356-b5db-4ccf-920e-3bc31edc9962-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683327    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-kubelet\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683351    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-ovnkube-script-lib\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683374    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-dbus\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:41.683837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683398    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683424    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-daemon-config\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683429    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-env-overrides\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683448    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-var-lib-openvswitch\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683240    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-slash\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683473    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-log-socket\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683491    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-kubelet\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683518    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6eeab356-b5db-4ccf-920e-3bc31edc9962-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683566    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-ovn-node-metrics-cert\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683602    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-cni-dir\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6eeab356-b5db-4ccf-920e-3bc31edc9962-system-cni-dir\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683651    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6eeab356-b5db-4ccf-920e-3bc31edc9962-cnibin\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683675    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6eeab356-b5db-4ccf-920e-3bc31edc9962-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683699    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683725    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-etc-kubernetes\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683751    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97e3d2a4-be3f-487f-b2a3-380e78fed6e1-tmp-dir\") pod \"node-resolver-q754g\" (UID: \"97e3d2a4-be3f-487f-b2a3-380e78fed6e1\") " pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683763    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-ovnkube-config\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.684730 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-cnibin\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/632a9615-66ff-47b8-b9e5-4357b516b064-serviceca\") pod \"node-ca-mgwk5\" (UID: \"632a9615-66ff-47b8-b9e5-4357b516b064\") " pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-cnibin\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683840    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-run-systemd\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683861    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-ovnkube-script-lib\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683880    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6eeab356-b5db-4ccf-920e-3bc31edc9962-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-etc-openvswitch\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683900    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-etc-openvswitch\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683909    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-run-systemd\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683920    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-run-openvswitch\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683932    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-var-lib-openvswitch\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683932    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-daemon-config\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-run-multus-certs\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6eeab356-b5db-4ccf-920e-3bc31edc9962-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683998    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhbdg\" (UniqueName: \"kubernetes.io/projected/6eeab356-b5db-4ccf-920e-3bc31edc9962-kube-api-access-zhbdg\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.684013    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684019    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-systemd-units\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684022    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6eeab356-b5db-4ccf-920e-3bc31edc9962-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.685395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684047    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-cni-bin\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.683885    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-log-socket\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684058    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-dbus\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:41.684064    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret podName:3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a nodeName:}" failed. No retries permitted until 2026-09-05 07:48:42.184049674 +0000 UTC m=+3.129371459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret") pod "global-pull-secret-syncer-6nw26" (UID: "3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684112    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6eeab356-b5db-4ccf-920e-3bc31edc9962-system-cni-dir\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684162    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6eeab356-b5db-4ccf-920e-3bc31edc9962-cnibin\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684189    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-kubelet-config\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/632a9615-66ff-47b8-b9e5-4357b516b064-serviceca\") pod \"node-ca-mgwk5\" (UID: \"632a9615-66ff-47b8-b9e5-4357b516b064\") " pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684214    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78shm\" (UniqueName: \"kubernetes.io/projected/4acfb359-d4f2-4e0a-a442-7871d3000d97-kube-api-access-78shm\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684222    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-etc-kubernetes\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684246    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-var-lib-kubelet\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684254    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-kubelet-config\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684268    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-hostroot\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684269    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684305    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-host-cni-bin\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684329    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-run-k8s-cni-cncf-io\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684346    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-conf-dir\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.685996 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4acfb359-d4f2-4e0a-a442-7871d3000d97-cni-binary-copy\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684419    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-var-lib-kubelet\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684463    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-hostroot\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684517    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-run-openvswitch\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684547    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-run-k8s-cni-cncf-io\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684560    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6eeab356-b5db-4ccf-920e-3bc31edc9962-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684575    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-host-run-multus-certs\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684599    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-systemd-units\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-conf-dir\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684708    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97e3d2a4-be3f-487f-b2a3-380e78fed6e1-tmp-dir\") pod \"node-resolver-q754g\" (UID: \"97e3d2a4-be3f-487f-b2a3-380e78fed6e1\") " pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4acfb359-d4f2-4e0a-a442-7871d3000d97-cni-binary-copy\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684864    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6eeab356-b5db-4ccf-920e-3bc31edc9962-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.684881    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4acfb359-d4f2-4e0a-a442-7871d3000d97-multus-cni-dir\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.686594 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.686490    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-ovn-node-metrics-cert\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.695290 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.695262    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzzwj\" (UniqueName: \"kubernetes.io/projected/97e3d2a4-be3f-487f-b2a3-380e78fed6e1-kube-api-access-hzzwj\") pod \"node-resolver-q754g\" (UID: \"97e3d2a4-be3f-487f-b2a3-380e78fed6e1\") " pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.695405 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.695324    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8wr4\" (UniqueName: \"kubernetes.io/projected/632a9615-66ff-47b8-b9e5-4357b516b064-kube-api-access-q8wr4\") pod \"node-ca-mgwk5\" (UID: \"632a9615-66ff-47b8-b9e5-4357b516b064\") " pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.695629 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.695599    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhbdg\" (UniqueName: \"kubernetes.io/projected/6eeab356-b5db-4ccf-920e-3bc31edc9962-kube-api-access-zhbdg\") pod \"multus-additional-cni-plugins-mvv55\" (UID: \"6eeab356-b5db-4ccf-920e-3bc31edc9962\") " pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.696401 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.696372    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78shm\" (UniqueName: \"kubernetes.io/projected/4acfb359-d4f2-4e0a-a442-7871d3000d97-kube-api-access-78shm\") pod \"multus-9tqz9\" (UID: \"4acfb359-d4f2-4e0a-a442-7871d3000d97\") " pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.696519 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.696494    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ljfm\" (UniqueName: \"kubernetes.io/projected/81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae-kube-api-access-7ljfm\") pod \"ovnkube-node-xp7vx\" (UID: \"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:41.774640 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.774614    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-shk85"
Sep 05 07:48:41.783456 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.783435    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:48:41.792165 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.792139    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5"
Sep 05 07:48:41.795917 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.795792    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pff8t"
Sep 05 07:48:41.802456 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.802435    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q754g"
Sep 05 07:48:41.808038 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.808022    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mgwk5"
Sep 05 07:48:41.815755 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.815735    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9tqz9"
Sep 05 07:48:41.820799 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.820781    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mvv55"
Sep 05 07:48:41.826645 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:41.826625    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:48:42.086795 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.086713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7t5\" (UniqueName: \"kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5\") pod \"network-check-target-7njm4\" (UID: \"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc\") " pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:42.086795 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.086750    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:42.086996 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:42.086874    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:42.086996 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:42.086877    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:42.086996 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:42.086895    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:42.086996 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:42.086904    2569 projected.go:196] Error preparing data for projected volume kube-api-access-9z7t5 for pod openshift-network-diagnostics/network-check-target-7njm4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:42.086996 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:42.086948    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5 podName:c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc nodeName:}" failed. No retries permitted until 2026-09-05 07:48:43.086932937 +0000 UTC m=+4.032254723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9z7t5" (UniqueName: "kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5") pod "network-check-target-7njm4" (UID: "c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:42.086996 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:42.086963    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs podName:6264509b-4382-45b2-a6f0-d8af26dd57f1 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:43.086955757 +0000 UTC m=+4.032277543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs") pod "network-metrics-daemon-bcxfm" (UID: "6264509b-4382-45b2-a6f0-d8af26dd57f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:42.187747 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.187716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:42.187917 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:42.187886    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:42.187965 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:42.187960    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret podName:3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a nodeName:}" failed. No retries permitted until 2026-09-05 07:48:43.187939914 +0000 UTC m=+4.133261710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret") pod "global-pull-secret-syncer-6nw26" (UID: "3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:42.457755 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:42.457729    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod632a9615_66ff_47b8_b9e5_4357b516b064.slice/crio-9b600fe810773e84c89e874426c0c5bb650bfdf3a725505d4ac4c84ac1281f89 WatchSource:0}: Error finding container 9b600fe810773e84c89e874426c0c5bb650bfdf3a725505d4ac4c84ac1281f89: Status 404 returned error can't find the container with id 9b600fe810773e84c89e874426c0c5bb650bfdf3a725505d4ac4c84ac1281f89
Sep 05 07:48:42.458730 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:42.458704    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e3d2a4_be3f_487f_b2a3_380e78fed6e1.slice/crio-672511e75b56760203127007fa75c0a3d2380be0cba6731b792f7246c58aecbd WatchSource:0}: Error finding container 672511e75b56760203127007fa75c0a3d2380be0cba6731b792f7246c58aecbd: Status 404 returned error can't find the container with id 672511e75b56760203127007fa75c0a3d2380be0cba6731b792f7246c58aecbd
Sep 05 07:48:42.459331 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:42.459266    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eeab356_b5db_4ccf_920e_3bc31edc9962.slice/crio-ebde3a285d8a41f5e8e1fe97149ad38834bcac3828818965f66490ffe4de4aae WatchSource:0}: Error finding container ebde3a285d8a41f5e8e1fe97149ad38834bcac3828818965f66490ffe4de4aae: Status 404 returned error can't find the container with id ebde3a285d8a41f5e8e1fe97149ad38834bcac3828818965f66490ffe4de4aae
Sep 05 07:48:42.460085 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:42.459889    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda07aa982_f058_45d8_b69d_f6f248285657.slice/crio-d7f7b1e5a80fd87b909ad176799d25eb6c52a84a566a59e9c107265a2669ce14 WatchSource:0}: Error finding container d7f7b1e5a80fd87b909ad176799d25eb6c52a84a566a59e9c107265a2669ce14: Status 404 returned error can't find the container with id d7f7b1e5a80fd87b909ad176799d25eb6c52a84a566a59e9c107265a2669ce14
Sep 05 07:48:42.460857 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:42.460818    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5387bb72_57d6_48da_8f99_365c22edc4dd.slice/crio-9b846f0c18976b8b9098f5dad7d9b2f27fa3d7d53658c0e90f97745dc1fa1c73 WatchSource:0}: Error finding container 9b846f0c18976b8b9098f5dad7d9b2f27fa3d7d53658c0e90f97745dc1fa1c73: Status 404 returned error can't find the container with id 9b846f0c18976b8b9098f5dad7d9b2f27fa3d7d53658c0e90f97745dc1fa1c73
Sep 05 07:48:42.464263 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:42.464244    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4acfb359_d4f2_4e0a_a442_7871d3000d97.slice/crio-8a3443459c4da392ade027df5ee94bd06f8cd2fb255303140ed4e3faf1188086 WatchSource:0}: Error finding container 8a3443459c4da392ade027df5ee94bd06f8cd2fb255303140ed4e3faf1188086: Status 404 returned error can't find the container with id 8a3443459c4da392ade027df5ee94bd06f8cd2fb255303140ed4e3faf1188086
Sep 05 07:48:42.465759 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:42.465736    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dfe1004_84bf_4d78_b813_e966d95dbd4c.slice/crio-28c96864b553cb18b0f3343a437247b24fcbc42fe108c8a3b9d4be4c2fce6c80 WatchSource:0}: Error finding container 28c96864b553cb18b0f3343a437247b24fcbc42fe108c8a3b9d4be4c2fce6c80: Status 404 returned error can't find the container with id 28c96864b553cb18b0f3343a437247b24fcbc42fe108c8a3b9d4be4c2fce6c80
Sep 05 07:48:42.467354 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:42.467331    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e9ce3b_89e2_488c_b4f3_4d4d4c2726ae.slice/crio-e2981f700dad62204d2c5fbfb29ab5dde82820716d4acdccf512f0c3b8c14384 WatchSource:0}: Error finding container e2981f700dad62204d2c5fbfb29ab5dde82820716d4acdccf512f0c3b8c14384: Status 404 returned error can't find the container with id e2981f700dad62204d2c5fbfb29ab5dde82820716d4acdccf512f0c3b8c14384
Sep 05 07:48:42.469212 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:48:42.469196    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe597340_d05d_40d4_90c8_7f4cb6fc6e01.slice/crio-45b660f15cd1010baf675fecf22ce6766269779f24c140e2280dcf706e8b3f2f WatchSource:0}: Error finding container 45b660f15cd1010baf675fecf22ce6766269779f24c140e2280dcf706e8b3f2f: Status 404 returned error can't find the container with id 45b660f15cd1010baf675fecf22ce6766269779f24c140e2280dcf706e8b3f2f
Sep 05 07:48:42.514987 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.514960    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 07:43:41 +0000 UTC" deadline="2028-02-08 00:12:15.48409395 +0000 UTC"
Sep 05 07:48:42.514987 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.514987    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12496h23m32.969110797s"
Sep 05 07:48:42.579042 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.579016    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" event={"ID":"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae","Type":"ContainerStarted","Data":"e2981f700dad62204d2c5fbfb29ab5dde82820716d4acdccf512f0c3b8c14384"}
Sep 05 07:48:42.580233 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.580210    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pff8t" event={"ID":"4dfe1004-84bf-4d78-b813-e966d95dbd4c","Type":"ContainerStarted","Data":"28c96864b553cb18b0f3343a437247b24fcbc42fe108c8a3b9d4be4c2fce6c80"}
Sep 05 07:48:42.583864 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.583839    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9tqz9" event={"ID":"4acfb359-d4f2-4e0a-a442-7871d3000d97","Type":"ContainerStarted","Data":"8a3443459c4da392ade027df5ee94bd06f8cd2fb255303140ed4e3faf1188086"}
Sep 05 07:48:42.584912 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.584890    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-shk85" event={"ID":"5387bb72-57d6-48da-8f99-365c22edc4dd","Type":"ContainerStarted","Data":"9b846f0c18976b8b9098f5dad7d9b2f27fa3d7d53658c0e90f97745dc1fa1c73"}
Sep 05 07:48:42.586610 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.586585    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v2j86" event={"ID":"a07aa982-f058-45d8-b69d-f6f248285657","Type":"ContainerStarted","Data":"d7f7b1e5a80fd87b909ad176799d25eb6c52a84a566a59e9c107265a2669ce14"}
Sep 05 07:48:42.587704 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.587683    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvv55" event={"ID":"6eeab356-b5db-4ccf-920e-3bc31edc9962","Type":"ContainerStarted","Data":"ebde3a285d8a41f5e8e1fe97149ad38834bcac3828818965f66490ffe4de4aae"}
Sep 05 07:48:42.588769 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.588749    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q754g" event={"ID":"97e3d2a4-be3f-487f-b2a3-380e78fed6e1","Type":"ContainerStarted","Data":"672511e75b56760203127007fa75c0a3d2380be0cba6731b792f7246c58aecbd"}
Sep 05 07:48:42.589798 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.589773    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mgwk5" event={"ID":"632a9615-66ff-47b8-b9e5-4357b516b064","Type":"ContainerStarted","Data":"9b600fe810773e84c89e874426c0c5bb650bfdf3a725505d4ac4c84ac1281f89"}
Sep 05 07:48:42.591655 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.591632    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal" event={"ID":"1120dc1be11d71eb0105a790715e7795","Type":"ContainerStarted","Data":"9ab69994b1e0130e0afe2b6e5f230547c73c3fe6f7ef8edbcc99aa4bbf4cb855"}
Sep 05 07:48:42.592806 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.592785    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5" event={"ID":"fe597340-d05d-40d4-90c8-7f4cb6fc6e01","Type":"ContainerStarted","Data":"45b660f15cd1010baf675fecf22ce6766269779f24c140e2280dcf706e8b3f2f"}
Sep 05 07:48:42.608381 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:42.608335    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-64.ec2.internal" podStartSLOduration=2.608323106 podStartE2EDuration="2.608323106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:42.607902449 +0000 UTC m=+3.553224254" watchObservedRunningTime="2026-09-05 07:48:42.608323106 +0000 UTC m=+3.553644910"
Sep 05 07:48:43.095441 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:43.094433    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:43.095441 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:43.094820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7t5\" (UniqueName: \"kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5\") pod \"network-check-target-7njm4\" (UID: \"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc\") " pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:43.095441 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.094967    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:43.095441 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.094991    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:43.095441 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.095005    2569 projected.go:196] Error preparing data for projected volume kube-api-access-9z7t5 for pod openshift-network-diagnostics/network-check-target-7njm4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:43.095441 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.095063    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5 podName:c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc nodeName:}" failed. No retries permitted until 2026-09-05 07:48:45.095045435 +0000 UTC m=+6.040367224 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9z7t5" (UniqueName: "kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5") pod "network-check-target-7njm4" (UID: "c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:43.095441 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.095386    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:43.095441 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.095442    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs podName:6264509b-4382-45b2-a6f0-d8af26dd57f1 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:45.095427252 +0000 UTC m=+6.040749050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs") pod "network-metrics-daemon-bcxfm" (UID: "6264509b-4382-45b2-a6f0-d8af26dd57f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:43.195963 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:43.195928    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:43.196133 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.196101    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:43.196211 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.196169    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret podName:3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a nodeName:}" failed. No retries permitted until 2026-09-05 07:48:45.196141698 +0000 UTC m=+6.141463495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret") pod "global-pull-secret-syncer-6nw26" (UID: "3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:43.515493 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:43.515355    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 07:43:41 +0000 UTC" deadline="2028-04-19 10:28:49.349209893 +0000 UTC"
Sep 05 07:48:43.515493 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:43.515409    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14210h40m5.833804253s"
Sep 05 07:48:43.565400 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:43.564627    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:43.565400 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.564793    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:48:43.565400 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:43.565256    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:43.565400 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.565351    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:48:43.568365 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:43.565883    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:43.568365 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:43.565997    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:48:43.604007 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:43.603969    2569 generic.go:356] "Generic (PLEG): container finished" podID="df5961174929c21de5075c8cdb7e20e2" containerID="4d2c5aa80d63519f5732bd4ab8bcda5504a7f818a659d93e399eb533dd7a64b3" exitCode=0
Sep 05 07:48:43.604951 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:43.604919    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal" event={"ID":"df5961174929c21de5075c8cdb7e20e2","Type":"ContainerDied","Data":"4d2c5aa80d63519f5732bd4ab8bcda5504a7f818a659d93e399eb533dd7a64b3"}
Sep 05 07:48:44.623573 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:44.623518    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal" event={"ID":"df5961174929c21de5075c8cdb7e20e2","Type":"ContainerStarted","Data":"c3de61168915a33794cc7e83a576f6f7635d5946153d9aa19fe4efd953d5b36a"}
Sep 05 07:48:45.114073 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:45.113288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7t5\" (UniqueName: \"kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5\") pod \"network-check-target-7njm4\" (UID: \"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc\") " pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:45.114073 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:45.113331    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:45.114073 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.113451    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:45.114073 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.113507    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs podName:6264509b-4382-45b2-a6f0-d8af26dd57f1 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:49.113488879 +0000 UTC m=+10.058810663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs") pod "network-metrics-daemon-bcxfm" (UID: "6264509b-4382-45b2-a6f0-d8af26dd57f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:45.114073 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.113917    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:45.114073 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.113937    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:45.114073 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.113949    2569 projected.go:196] Error preparing data for projected volume kube-api-access-9z7t5 for pod openshift-network-diagnostics/network-check-target-7njm4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:45.114073 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.113992    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5 podName:c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc nodeName:}" failed. No retries permitted until 2026-09-05 07:48:49.11397812 +0000 UTC m=+10.059299908 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9z7t5" (UniqueName: "kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5") pod "network-check-target-7njm4" (UID: "c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:45.215053 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:45.214480    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:45.215053 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.214680    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:45.215053 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.214735    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret podName:3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a nodeName:}" failed. No retries permitted until 2026-09-05 07:48:49.214717604 +0000 UTC m=+10.160039393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret") pod "global-pull-secret-syncer-6nw26" (UID: "3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:45.565398 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:45.565326    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:45.565569 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:45.565326    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:45.565569 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.565455    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:48:45.565569 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:45.565326    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:45.565737 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.565562    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:48:45.565737 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:45.565657    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:48:47.564715 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:47.564686    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:47.565169 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:47.564820    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:48:47.565235 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:47.565179    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:47.565290 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:47.565243    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:48:47.565391 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:47.565371    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:47.565480 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:47.565463    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:48:49.144333 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:49.144295    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7t5\" (UniqueName: \"kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5\") pod \"network-check-target-7njm4\" (UID: \"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc\") " pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:49.144831 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:49.144345    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:49.144831 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.144481    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:49.144831 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.144488    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:49.144831 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.144512    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:49.144831 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.144544    2569 projected.go:196] Error preparing data for projected volume kube-api-access-9z7t5 for pod openshift-network-diagnostics/network-check-target-7njm4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:49.144831 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.144557    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs podName:6264509b-4382-45b2-a6f0-d8af26dd57f1 nodeName:}" failed. No retries permitted until 2026-09-05 07:48:57.144521304 +0000 UTC m=+18.089843090 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs") pod "network-metrics-daemon-bcxfm" (UID: "6264509b-4382-45b2-a6f0-d8af26dd57f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:49.144831 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.144594    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5 podName:c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc nodeName:}" failed. No retries permitted until 2026-09-05 07:48:57.144576935 +0000 UTC m=+18.089898721 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9z7t5" (UniqueName: "kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5") pod "network-check-target-7njm4" (UID: "c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:49.245716 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:49.245684    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:49.245867 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.245856    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:49.245932 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.245909    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret podName:3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a nodeName:}" failed. No retries permitted until 2026-09-05 07:48:57.245896006 +0000 UTC m=+18.191218009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret") pod "global-pull-secret-syncer-6nw26" (UID: "3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:49.566521 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:49.564000    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:49.566521 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.564134    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:48:49.566521 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:49.564233    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:49.566521 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:49.564264    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:49.566521 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.564356    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:48:49.566521 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:49.565483    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:48:51.567520 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:51.567108    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:51.567520 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:51.567208    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:51.567520 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:51.567220    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:48:51.567520 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:51.567245    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:51.567520 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:51.567336    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:48:51.567520 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:51.567406    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:48:53.568054 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:53.568027    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:53.568521 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:53.568079    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:53.568521 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:53.568084    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:53.568521 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:53.568176    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:48:53.568521 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:53.568289    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:48:53.568521 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:53.568391    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:48:55.564399 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:55.564360    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:55.564399 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:55.564386    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:55.564399 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:55.564366    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:55.564963 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:55.564488    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:48:55.564963 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:55.564600    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:48:55.564963 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:55.564722    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:48:57.205906 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:57.205873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7t5\" (UniqueName: \"kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5\") pod \"network-check-target-7njm4\" (UID: \"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc\") " pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:57.205906 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:57.205909    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:57.206504 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.206020    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:57.206504 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.206033    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:48:57.206504 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.206059    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:48:57.206504 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.206073    2569 projected.go:196] Error preparing data for projected volume kube-api-access-9z7t5 for pod openshift-network-diagnostics/network-check-target-7njm4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:57.206504 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.206081    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs podName:6264509b-4382-45b2-a6f0-d8af26dd57f1 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:13.206064398 +0000 UTC m=+34.151386201 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs") pod "network-metrics-daemon-bcxfm" (UID: "6264509b-4382-45b2-a6f0-d8af26dd57f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:48:57.206504 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.206114    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5 podName:c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc nodeName:}" failed. No retries permitted until 2026-09-05 07:49:13.206101015 +0000 UTC m=+34.151422801 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9z7t5" (UniqueName: "kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5") pod "network-check-target-7njm4" (UID: "c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:48:57.306803 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:57.306770    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:57.306961 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.306933    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:57.307013 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.306998    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret podName:3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a nodeName:}" failed. No retries permitted until 2026-09-05 07:49:13.306984564 +0000 UTC m=+34.252306347 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret") pod "global-pull-secret-syncer-6nw26" (UID: "3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:48:57.564182 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:57.564105    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:57.564346 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:57.564105    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:57.564346 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.564237    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:48:57.564346 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:57.564113    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:57.564346 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.564321    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:48:57.564543 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:57.564428    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:48:59.568099 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:59.568071    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:48:59.568569 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:59.568177    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:48:59.568569 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:59.568185    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:48:59.568569 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:59.568270    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:48:59.568569 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:48:59.568184    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:48:59.568569 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:48:59.568361    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:00.654951 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:00.654926    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v2j86" event={"ID":"a07aa982-f058-45d8-b69d-f6f248285657","Type":"ContainerStarted","Data":"8ec1907a3905b329ee28ae80a54481e9bc378ca1e5e6147cd59d2890b7686d30"}
Sep 05 07:49:01.564327 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.563982    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:01.564563 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.563982    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:01.564563 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:01.564388    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:01.564563 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.564037    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:01.564563 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:01.564456    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:01.564563 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:01.564513    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:01.658671 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.658640    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5" event={"ID":"fe597340-d05d-40d4-90c8-7f4cb6fc6e01","Type":"ContainerStarted","Data":"b1f2609ce323759863c553a5e78bee9312d71ca7fc916cd7424e693e8cce0570"}
Sep 05 07:49:01.661316 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.661296    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 07:49:01.661627 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.661605    2569 generic.go:356] "Generic (PLEG): container finished" podID="81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae" containerID="a5826f27e91b53ca556016fe9ffe8958d12bf023457410be54fe396b7eff4f86" exitCode=1
Sep 05 07:49:01.661747 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.661675    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" event={"ID":"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae","Type":"ContainerStarted","Data":"e52165574206941eee2670ce2cbdd219921c38b8a9f938b45c5d7c47d305e794"}
Sep 05 07:49:01.661747 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.661736    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" event={"ID":"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae","Type":"ContainerStarted","Data":"1066da917ad5a20075a16e1b533e4b7bb4ed813c5d921896399d1e6fea4a7491"}
Sep 05 07:49:01.661817 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.661769    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" event={"ID":"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae","Type":"ContainerStarted","Data":"85e4345446c0cc4f26c85b4442a2e5b431541ca69ccf5529748957749b9e60da"}
Sep 05 07:49:01.661817 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.661797    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" event={"ID":"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae","Type":"ContainerStarted","Data":"c9fe7b8a7a8c17c409c2076a9e035fcf04fea37ae85930f1af8a60970d3cc6f8"}
Sep 05 07:49:01.661901 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.661832    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" event={"ID":"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae","Type":"ContainerDied","Data":"a5826f27e91b53ca556016fe9ffe8958d12bf023457410be54fe396b7eff4f86"}
Sep 05 07:49:01.661901 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.661863    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" event={"ID":"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae","Type":"ContainerStarted","Data":"11800d2a4803a75080a74fb9715261930607a15d18fe98826ac211a4bd92d20a"}
Sep 05 07:49:01.663039 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.663014    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pff8t" event={"ID":"4dfe1004-84bf-4d78-b813-e966d95dbd4c","Type":"ContainerStarted","Data":"cc0e42beb78987b2a5018dd0545f033e9f92d081ecb3bfc194ead559b05ee70b"}
Sep 05 07:49:01.664496 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.664475    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9tqz9" event={"ID":"4acfb359-d4f2-4e0a-a442-7871d3000d97","Type":"ContainerStarted","Data":"0738e1f4f6ffcbd8ca6e99197b99ec17976cf8ff688004405f3ac083a6a79e81"}
Sep 05 07:49:01.665935 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.665911    2569 generic.go:356] "Generic (PLEG): container finished" podID="6eeab356-b5db-4ccf-920e-3bc31edc9962" containerID="696d2c25163d8c3a4817b582e16f9883b27ec242ea7ae1bc6d4048c782244b4c" exitCode=0
Sep 05 07:49:01.666031 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.665990    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvv55" event={"ID":"6eeab356-b5db-4ccf-920e-3bc31edc9962","Type":"ContainerDied","Data":"696d2c25163d8c3a4817b582e16f9883b27ec242ea7ae1bc6d4048c782244b4c"}
Sep 05 07:49:01.667491 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.667432    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q754g" event={"ID":"97e3d2a4-be3f-487f-b2a3-380e78fed6e1","Type":"ContainerStarted","Data":"d24ea5fe3c7755b4ffaba60150c92d1c161d953ff17d55214c7abc23c5c51e62"}
Sep 05 07:49:01.669022 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.668994    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mgwk5" event={"ID":"632a9615-66ff-47b8-b9e5-4357b516b064","Type":"ContainerStarted","Data":"57c5a9d55b254ad7028b2997432e689528d2558a4f283049c3b41adda07680b1"}
Sep 05 07:49:01.682383 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.682337    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pff8t" podStartSLOduration=4.612856483 podStartE2EDuration="22.68232113s" totalImagesPullingTime="18.069464647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:01.682049823 +0000 UTC m=+22.627371628" watchObservedRunningTime="2026-09-05 07:49:01.68232113 +0000 UTC m=+22.627642935"
Sep 05 07:49:01.682928 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.682901    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-64.ec2.internal" podStartSLOduration=21.682895078 podStartE2EDuration="21.682895078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:48:44.640672987 +0000 UTC m=+5.585994796" watchObservedRunningTime="2026-09-05 07:49:01.682895078 +0000 UTC m=+22.628216883"
Sep 05 07:49:01.714402 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.714368    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v2j86" podStartSLOduration=12.140021556 podStartE2EDuration="22.714354694s" totalImagesPullingTime="10.574333138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:01.698566933 +0000 UTC m=+22.643888734" watchObservedRunningTime="2026-09-05 07:49:01.714354694 +0000 UTC m=+22.659676490"
Sep 05 07:49:01.714605 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.714585    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-q754g" podStartSLOduration=4.669226024 podStartE2EDuration="22.714577809s" totalImagesPullingTime="18.045351785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:01.714559016 +0000 UTC m=+22.659880821" watchObservedRunningTime="2026-09-05 07:49:01.714577809 +0000 UTC m=+22.659899613"
Sep 05 07:49:01.729245 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.729210    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mgwk5" podStartSLOduration=4.682620519 podStartE2EDuration="22.729199999s" totalImagesPullingTime="18.04657948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:01.729147602 +0000 UTC m=+22.674469406" watchObservedRunningTime="2026-09-05 07:49:01.729199999 +0000 UTC m=+22.674521795"
Sep 05 07:49:01.749079 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:01.749048    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9tqz9" podStartSLOduration=4.307202887 podStartE2EDuration="22.749038131s" totalImagesPullingTime="18.441835244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:01.748949471 +0000 UTC m=+22.694271275" watchObservedRunningTime="2026-09-05 07:49:01.749038131 +0000 UTC m=+22.694359935"
Sep 05 07:49:02.036258 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:02.036230    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 05 07:49:02.537398 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:02.537291    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-05T07:49:02.036253256Z","UUID":"32607fd9-3b5a-4c21-abe3-fafd1e9ed8ef","Handler":null,"Name":"","Endpoint":""}
Sep 05 07:49:02.538831 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:02.538801    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 05 07:49:02.538948 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:02.538842    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 05 07:49:02.674485 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:02.674446    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5" event={"ID":"fe597340-d05d-40d4-90c8-7f4cb6fc6e01","Type":"ContainerStarted","Data":"3af93bb742940394f48995eb4e98354aa6ce690fe72cae2721260e1b9b52f5a0"}
Sep 05 07:49:02.676514 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:02.676486    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-shk85" event={"ID":"5387bb72-57d6-48da-8f99-365c22edc4dd","Type":"ContainerStarted","Data":"cf9fdc7fcf7d34cf7d2711d86b8b34c8c83570f900dd189da47c49d29b9410d9"}
Sep 05 07:49:02.693952 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:02.693910    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-shk85" podStartSLOduration=5.650762411 podStartE2EDuration="23.693897389s" totalImagesPullingTime="18.043134978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:02.693464246 +0000 UTC m=+23.638786050" watchObservedRunningTime="2026-09-05 07:49:02.693897389 +0000 UTC m=+23.639219194"
Sep 05 07:49:03.564162 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:03.564126    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:03.564321 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:03.564136    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:03.564321 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:03.564265    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:03.564321 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:03.564270    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:03.564480 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:03.564351    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:03.564480 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:03.564439    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:03.681441 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:03.681409    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5" event={"ID":"fe597340-d05d-40d4-90c8-7f4cb6fc6e01","Type":"ContainerStarted","Data":"269635ba67dbf46c2cbfbc2ad9b3d9cd8e5fbd375cc28ce29279f222efd533a3"}
Sep 05 07:49:03.686022 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:03.685999    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 07:49:03.686462 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:03.686437    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" event={"ID":"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae","Type":"ContainerStarted","Data":"57730fffbaccf1ea02c2a92e14bfd957d2a8e2ea4a83a669edc14a1c9f4e8664"}
Sep 05 07:49:03.699577 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:03.699509    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf4l5" podStartSLOduration=4.318123309 podStartE2EDuration="24.699492453s" totalImagesPullingTime="20.381369144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:03.699026245 +0000 UTC m=+24.644348064" watchObservedRunningTime="2026-09-05 07:49:03.699492453 +0000 UTC m=+24.644814259"
Sep 05 07:49:03.811654 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:03.811577    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:49:03.812339 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:03.812316    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:49:05.565023 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:05.564806    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:05.565433 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:05.564824    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:05.565433 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:05.565107    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:05.565433 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:05.565211    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:05.565433 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:05.564884    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:05.565433 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:05.565307    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:06.696605 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:06.696579    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 07:49:06.697476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:06.696992    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" event={"ID":"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae","Type":"ContainerStarted","Data":"2ea90c02a2785f42bb88a9fdbf2a6079705d7ca424f171124276d98258ef1ae6"}
Sep 05 07:49:06.697476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:06.697254    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:49:06.697476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:06.697387    2569 scope.go:117] "RemoveContainer" containerID="a5826f27e91b53ca556016fe9ffe8958d12bf023457410be54fe396b7eff4f86"
Sep 05 07:49:06.697476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:06.697405    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:49:06.697476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:06.697461    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:49:06.699109 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:06.699089    2569 generic.go:356] "Generic (PLEG): container finished" podID="6eeab356-b5db-4ccf-920e-3bc31edc9962" containerID="a9bc4f577c8d457ba7b3edabb1ec6c835131e03405d23562b88c5edeb5377935" exitCode=0
Sep 05 07:49:06.699202 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:06.699115    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvv55" event={"ID":"6eeab356-b5db-4ccf-920e-3bc31edc9962","Type":"ContainerDied","Data":"a9bc4f577c8d457ba7b3edabb1ec6c835131e03405d23562b88c5edeb5377935"}
Sep 05 07:49:06.714873 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:06.714836    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:49:06.719966 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:06.719945    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:49:07.564468 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.564440    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:07.564680 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:07.564656    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:07.564771 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.564447    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:07.564822 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:07.564775    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:07.564822 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.564440    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:07.564919 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:07.564856    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:07.705518 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.705446    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 07:49:07.705871 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.705843    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" event={"ID":"81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae","Type":"ContainerStarted","Data":"93a3e62a0c290810b038ee53b0779d5a6d591b7a1cafb2a33915668c55d77f16"}
Sep 05 07:49:07.708045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.708022    2569 generic.go:356] "Generic (PLEG): container finished" podID="6eeab356-b5db-4ccf-920e-3bc31edc9962" containerID="b06b0b13eea3f8aabcdad56369c81aa6668e828a83a5ea6f55d6b723768f9b90" exitCode=0
Sep 05 07:49:07.708155 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.708055    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvv55" event={"ID":"6eeab356-b5db-4ccf-920e-3bc31edc9962","Type":"ContainerDied","Data":"b06b0b13eea3f8aabcdad56369c81aa6668e828a83a5ea6f55d6b723768f9b90"}
Sep 05 07:49:07.733596 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.733549    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx" podStartSLOduration=10.662025862 podStartE2EDuration="28.733509248s" totalImagesPullingTime="18.071483386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:07.731654188 +0000 UTC m=+28.676975995" watchObservedRunningTime="2026-09-05 07:49:07.733509248 +0000 UTC m=+28.678831053"
Sep 05 07:49:07.887506 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.887469    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6nw26"]
Sep 05 07:49:07.887732 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.887613    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:07.887800 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:07.887724    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:07.888177 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.888155    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7njm4"]
Sep 05 07:49:07.888371 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.888239    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:07.888371 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:07.888326    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:07.888971 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.888948    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bcxfm"]
Sep 05 07:49:07.889080 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:07.889029    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:07.889148 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:07.889117    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:08.712870 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:08.712780    2569 generic.go:356] "Generic (PLEG): container finished" podID="6eeab356-b5db-4ccf-920e-3bc31edc9962" containerID="5c12c599d167a8745ad6ddba545a6174ec4ae7cb6af41c72342af4e7dbc5b5ed" exitCode=0
Sep 05 07:49:08.713200 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:08.712870    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvv55" event={"ID":"6eeab356-b5db-4ccf-920e-3bc31edc9962","Type":"ContainerDied","Data":"5c12c599d167a8745ad6ddba545a6174ec4ae7cb6af41c72342af4e7dbc5b5ed"}
Sep 05 07:49:09.564731 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:09.564700    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:09.564878 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:09.564857    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:09.564926 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:09.564884    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:09.564926 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:09.564870    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:09.565005 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:09.564971    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:09.565046 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:09.565022    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:09.591503 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:09.591476    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q754g_97e3d2a4-be3f-487f-b2a3-380e78fed6e1/dns-node-resolver/0.log"
Sep 05 07:49:10.566955 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:10.566752    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mgwk5_632a9615-66ff-47b8-b9e5-4357b516b064/node-ca/0.log"
Sep 05 07:49:10.914786 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:10.914748    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:49:10.914975 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:10.914931    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 05 07:49:10.915428 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:10.915399    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v2j86"
Sep 05 07:49:11.564453 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:11.564419    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:11.564638 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:11.564431    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:11.564638 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:11.564561    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:11.564638 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:11.564577    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:11.564807 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:11.564648    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:11.564807 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:11.564726    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:13.218934 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:13.218899    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7t5\" (UniqueName: \"kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5\") pod \"network-check-target-7njm4\" (UID: \"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc\") " pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:13.218934 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:13.218937    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:13.219424 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.219032    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:49:13.219424 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.219076    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 07:49:13.219424 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.219098    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 07:49:13.219424 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.219111    2569 projected.go:196] Error preparing data for projected volume kube-api-access-9z7t5 for pod openshift-network-diagnostics/network-check-target-7njm4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:49:13.219424 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.219081    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs podName:6264509b-4382-45b2-a6f0-d8af26dd57f1 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:45.219068571 +0000 UTC m=+66.164390358 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs") pod "network-metrics-daemon-bcxfm" (UID: "6264509b-4382-45b2-a6f0-d8af26dd57f1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 07:49:13.219424 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.219173    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5 podName:c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc nodeName:}" failed. No retries permitted until 2026-09-05 07:49:45.219153266 +0000 UTC m=+66.164475053 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9z7t5" (UniqueName: "kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5") pod "network-check-target-7njm4" (UID: "c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 07:49:13.320234 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:13.320208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:13.320416 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.320353    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 07:49:13.320416 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.320414    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret podName:3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a nodeName:}" failed. No retries permitted until 2026-09-05 07:49:45.320399649 +0000 UTC m=+66.265721432 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret") pod "global-pull-secret-syncer-6nw26" (UID: "3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 07:49:13.564423 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:13.564344    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:13.564423 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:13.564377    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:13.564628 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:13.564350    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:13.564628 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.564453    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:13.564628 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.564551    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:13.564721 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:13.564645    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:13.726451 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:13.726419    2569 generic.go:356] "Generic (PLEG): container finished" podID="6eeab356-b5db-4ccf-920e-3bc31edc9962" containerID="0c74792f7ae068b29da7c370f2ee289713a841a96e63ad9c32404b7a8f408973" exitCode=0
Sep 05 07:49:13.726646 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:13.726489    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvv55" event={"ID":"6eeab356-b5db-4ccf-920e-3bc31edc9962","Type":"ContainerDied","Data":"0c74792f7ae068b29da7c370f2ee289713a841a96e63ad9c32404b7a8f408973"}
Sep 05 07:49:14.732169 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:14.732121    2569 generic.go:356] "Generic (PLEG): container finished" podID="6eeab356-b5db-4ccf-920e-3bc31edc9962" containerID="638546af9ae2df578e6fe756128851cafb1c711bb851951ed68c16378b348802" exitCode=0
Sep 05 07:49:14.732656 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:14.732203    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvv55" event={"ID":"6eeab356-b5db-4ccf-920e-3bc31edc9962","Type":"ContainerDied","Data":"638546af9ae2df578e6fe756128851cafb1c711bb851951ed68c16378b348802"}
Sep 05 07:49:15.564367 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:15.564324    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:15.564367 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:15.564364    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:15.564627 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:15.564328    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:15.564627 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:15.564441    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:15.564627 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:15.564575    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:15.564627 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:15.564612    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:15.737919 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:15.737880    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvv55" event={"ID":"6eeab356-b5db-4ccf-920e-3bc31edc9962","Type":"ContainerStarted","Data":"d27a369a65602e7b9312203c18eb3d092d8e1be35b606444a5f0acec4d544ce4"}
Sep 05 07:49:15.764243 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:15.764196    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mvv55" podStartSLOduration=8.647269918 podStartE2EDuration="36.764181356s" totalImagesPullingTime="28.116911438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:15.763861185 +0000 UTC m=+36.709182990" watchObservedRunningTime="2026-09-05 07:49:15.764181356 +0000 UTC m=+36.709503160"
Sep 05 07:49:17.564641 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:17.564607    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:17.565109 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:17.564610    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:17.565109 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:17.564701    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:17.565109 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:17.564607    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:17.565109 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:17.564777    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:17.565109 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:17.564835    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:19.565487 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:19.565451    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:19.565997 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:19.565553    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:19.565997 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:19.565569    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:19.565997 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:19.565696    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:19.565997 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:19.565759    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:19.565997 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:19.565690    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:21.564719 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:21.564692    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:21.564719 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:21.564716    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:21.565156 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:21.564721    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:21.565156 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:21.564780    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:21.565156 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:21.564863    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:21.565156 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:21.564931    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:23.567626 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:23.567602    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:23.568006 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:23.567603    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:23.568006 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:23.567694    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:23.568006 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:23.567753    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:23.568006 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:23.567602    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:23.568006 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:23.567833    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:25.564499 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:25.564467    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:25.564939 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:25.564467    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:25.564939 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:25.564599    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:25.564939 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:25.564473    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:25.564939 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:25.564661    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:25.564939 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:25.564764    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:27.564280 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:27.564248    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:27.564280 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:27.564275    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:27.564691 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:27.564255    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:27.564691 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:27.564343    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:27.564691 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:27.564417    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:27.564691 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:27.564477    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:29.565017 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:29.564989    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:29.565481 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:29.565067    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:29.565481 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:29.565068    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:29.565481 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:29.565143    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:29.565481 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:29.565154    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:29.565481 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:29.565261    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:31.564007 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:31.563973    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:31.564465 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:31.563974    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:31.564465 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:31.564064    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7njm4" podUID="c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc"
Sep 05 07:49:31.564465 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:31.564093    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:31.564465 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:31.564178    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bcxfm" podUID="6264509b-4382-45b2-a6f0-d8af26dd57f1"
Sep 05 07:49:31.564465 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:31.564240    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6nw26" podUID="3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a"
Sep 05 07:49:32.881970 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.881941    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-64.ec2.internal" event="NodeReady"
Sep 05 07:49:32.882320 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.882064    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 05 07:49:32.982243 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.982210    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vtcdn"]
Sep 05 07:49:32.984072 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.984056    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ph7k8"]
Sep 05 07:49:32.984210 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.984194    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:32.985628 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.985607    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:32.988471 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.988454    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 05 07:49:32.988577 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.988517    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 05 07:49:32.988652 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.988595    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 05 07:49:32.988756 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.988737    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 05 07:49:32.989009 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.988993    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 05 07:49:32.989883 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.989866    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 05 07:49:32.989985 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.989901    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gx2xl\""
Sep 05 07:49:32.998930 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:32.998914    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t6wxv\""
Sep 05 07:49:33.021084 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.021052    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ph7k8"]
Sep 05 07:49:33.054087 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.054054    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bbfb7b6c-970f-4d09-8eaf-e4944b723741-data-volume\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.054087 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.054090    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a0f2752-8a69-451c-a57d-c38242e3350d-tmp-dir\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.054295 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.054122    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bbfb7b6c-970f-4d09-8eaf-e4944b723741-crio-socket\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.054295 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.054180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86bmp\" (UniqueName: \"kubernetes.io/projected/bbfb7b6c-970f-4d09-8eaf-e4944b723741-kube-api-access-86bmp\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.054295 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.054246    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a0f2752-8a69-451c-a57d-c38242e3350d-config-volume\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.054440 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.054299    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbfb7b6c-970f-4d09-8eaf-e4944b723741-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.054440 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.054325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbfb7b6c-970f-4d09-8eaf-e4944b723741-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.054440 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.054345    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a0f2752-8a69-451c-a57d-c38242e3350d-metrics-tls\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.054440 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.054389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj95p\" (UniqueName: \"kubernetes.io/projected/1a0f2752-8a69-451c-a57d-c38242e3350d-kube-api-access-vj95p\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.055746 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.055725    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vtcdn"]
Sep 05 07:49:33.074651 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.074628    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ksjgr"]
Sep 05 07:49:33.076207 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.076194    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ksjgr"
Sep 05 07:49:33.079097 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.079077    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 05 07:49:33.080347 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.080326    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qqkv2\""
Sep 05 07:49:33.080347 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.080343    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 05 07:49:33.080500 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.080392    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 05 07:49:33.087824 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.087805    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ksjgr"]
Sep 05 07:49:33.154669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.154602    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bbfb7b6c-970f-4d09-8eaf-e4944b723741-crio-socket\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.154669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.154632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86bmp\" (UniqueName: \"kubernetes.io/projected/bbfb7b6c-970f-4d09-8eaf-e4944b723741-kube-api-access-86bmp\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.154669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.154649    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a0f2752-8a69-451c-a57d-c38242e3350d-config-volume\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.154669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.154671    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbfb7b6c-970f-4d09-8eaf-e4944b723741-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.154899 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.154728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bbfb7b6c-970f-4d09-8eaf-e4944b723741-crio-socket\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.154899 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.154843    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbfb7b6c-970f-4d09-8eaf-e4944b723741-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.154899 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.154862    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a0f2752-8a69-451c-a57d-c38242e3350d-metrics-tls\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.154899 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.154891    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj95p\" (UniqueName: \"kubernetes.io/projected/1a0f2752-8a69-451c-a57d-c38242e3350d-kube-api-access-vj95p\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.155086 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.154919    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjg9p\" (UniqueName: \"kubernetes.io/projected/735870a0-b9d2-4e05-8e45-7e01af14f5a3-kube-api-access-rjg9p\") pod \"ingress-canary-ksjgr\" (UID: \"735870a0-b9d2-4e05-8e45-7e01af14f5a3\") " pod="openshift-ingress-canary/ingress-canary-ksjgr"
Sep 05 07:49:33.155086 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.155060    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/735870a0-b9d2-4e05-8e45-7e01af14f5a3-cert\") pod \"ingress-canary-ksjgr\" (UID: \"735870a0-b9d2-4e05-8e45-7e01af14f5a3\") " pod="openshift-ingress-canary/ingress-canary-ksjgr"
Sep 05 07:49:33.155187 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.155109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bbfb7b6c-970f-4d09-8eaf-e4944b723741-data-volume\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.155187 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.155132    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a0f2752-8a69-451c-a57d-c38242e3350d-tmp-dir\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.155291 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.155265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a0f2752-8a69-451c-a57d-c38242e3350d-config-volume\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.155350 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.155284    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbfb7b6c-970f-4d09-8eaf-e4944b723741-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.155440 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.155408    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a0f2752-8a69-451c-a57d-c38242e3350d-tmp-dir\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.155564 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.155446    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bbfb7b6c-970f-4d09-8eaf-e4944b723741-data-volume\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.158814 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.158798    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a0f2752-8a69-451c-a57d-c38242e3350d-metrics-tls\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.158881 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.158823    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbfb7b6c-970f-4d09-8eaf-e4944b723741-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.163881 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.163854    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86bmp\" (UniqueName: \"kubernetes.io/projected/bbfb7b6c-970f-4d09-8eaf-e4944b723741-kube-api-access-86bmp\") pod \"insights-runtime-extractor-ph7k8\" (UID: \"bbfb7b6c-970f-4d09-8eaf-e4944b723741\") " pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.164999 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.164980    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj95p\" (UniqueName: \"kubernetes.io/projected/1a0f2752-8a69-451c-a57d-c38242e3350d-kube-api-access-vj95p\") pod \"dns-default-vtcdn\" (UID: \"1a0f2752-8a69-451c-a57d-c38242e3350d\") " pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.255658 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.255620    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg9p\" (UniqueName: \"kubernetes.io/projected/735870a0-b9d2-4e05-8e45-7e01af14f5a3-kube-api-access-rjg9p\") pod \"ingress-canary-ksjgr\" (UID: \"735870a0-b9d2-4e05-8e45-7e01af14f5a3\") " pod="openshift-ingress-canary/ingress-canary-ksjgr"
Sep 05 07:49:33.255658 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.255663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/735870a0-b9d2-4e05-8e45-7e01af14f5a3-cert\") pod \"ingress-canary-ksjgr\" (UID: \"735870a0-b9d2-4e05-8e45-7e01af14f5a3\") " pod="openshift-ingress-canary/ingress-canary-ksjgr"
Sep 05 07:49:33.258187 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.258168    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/735870a0-b9d2-4e05-8e45-7e01af14f5a3-cert\") pod \"ingress-canary-ksjgr\" (UID: \"735870a0-b9d2-4e05-8e45-7e01af14f5a3\") " pod="openshift-ingress-canary/ingress-canary-ksjgr"
Sep 05 07:49:33.264489 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.264466    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjg9p\" (UniqueName: \"kubernetes.io/projected/735870a0-b9d2-4e05-8e45-7e01af14f5a3-kube-api-access-rjg9p\") pod \"ingress-canary-ksjgr\" (UID: \"735870a0-b9d2-4e05-8e45-7e01af14f5a3\") " pod="openshift-ingress-canary/ingress-canary-ksjgr"
Sep 05 07:49:33.294396 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.294361    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:33.298756 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.298732    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ph7k8"
Sep 05 07:49:33.384463 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.384434    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ksjgr"
Sep 05 07:49:33.565056 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.565026    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:33.565221 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.565028    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:33.565377 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.565028    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:33.568330 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.568312    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 05 07:49:33.569454 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.569437    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 05 07:49:33.569595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.569473    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g9xp6\""
Sep 05 07:49:33.569595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.569485    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 05 07:49:33.569595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.569551    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 05 07:49:33.569595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.569473    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hp5x9\""
Sep 05 07:49:33.579127 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.579102    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vtcdn"]
Sep 05 07:49:33.582392 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.582367    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ph7k8"]
Sep 05 07:49:33.582501 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:33.582374    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0f2752_8a69_451c_a57d_c38242e3350d.slice/crio-adf93342bb6a607aa0284c9c0c56f904baf074785e2d789264ca61c88817ed95 WatchSource:0}: Error finding container adf93342bb6a607aa0284c9c0c56f904baf074785e2d789264ca61c88817ed95: Status 404 returned error can't find the container with id adf93342bb6a607aa0284c9c0c56f904baf074785e2d789264ca61c88817ed95
Sep 05 07:49:33.586276 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:33.585441    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbfb7b6c_970f_4d09_8eaf_e4944b723741.slice/crio-977fa9f237b8cb39991d5ff7dd6943282aeaa6c51743799a9d10f3382f218749 WatchSource:0}: Error finding container 977fa9f237b8cb39991d5ff7dd6943282aeaa6c51743799a9d10f3382f218749: Status 404 returned error can't find the container with id 977fa9f237b8cb39991d5ff7dd6943282aeaa6c51743799a9d10f3382f218749
Sep 05 07:49:33.652577 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.652467    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ksjgr"]
Sep 05 07:49:33.655414 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:33.655393    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735870a0_b9d2_4e05_8e45_7e01af14f5a3.slice/crio-f3881045e74b457e226b15b1b8e32bee14256f8229b94f07206840643d1b82bb WatchSource:0}: Error finding container f3881045e74b457e226b15b1b8e32bee14256f8229b94f07206840643d1b82bb: Status 404 returned error can't find the container with id f3881045e74b457e226b15b1b8e32bee14256f8229b94f07206840643d1b82bb
Sep 05 07:49:33.776693 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.776603    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ksjgr" event={"ID":"735870a0-b9d2-4e05-8e45-7e01af14f5a3","Type":"ContainerStarted","Data":"f3881045e74b457e226b15b1b8e32bee14256f8229b94f07206840643d1b82bb"}
Sep 05 07:49:33.778108 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.778081    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ph7k8" event={"ID":"bbfb7b6c-970f-4d09-8eaf-e4944b723741","Type":"ContainerStarted","Data":"4a7b9057a58be73087c8cf054000e0d22789a578b52c33156afa9c46d5a87634"}
Sep 05 07:49:33.778206 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.778131    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ph7k8" event={"ID":"bbfb7b6c-970f-4d09-8eaf-e4944b723741","Type":"ContainerStarted","Data":"977fa9f237b8cb39991d5ff7dd6943282aeaa6c51743799a9d10f3382f218749"}
Sep 05 07:49:33.779161 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:33.779142    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vtcdn" event={"ID":"1a0f2752-8a69-451c-a57d-c38242e3350d","Type":"ContainerStarted","Data":"adf93342bb6a607aa0284c9c0c56f904baf074785e2d789264ca61c88817ed95"}
Sep 05 07:49:34.082985 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.082950    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74f48649c8-7b27q"]
Sep 05 07:49:34.084964 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.084942    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.087882 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.087842    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 05 07:49:34.088005 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.087986    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 05 07:49:34.089512 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.089493    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 05 07:49:34.089636 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.089570    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 05 07:49:34.089696 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.089493    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 05 07:49:34.089696 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.089651    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 05 07:49:34.090023 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.089887    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 05 07:49:34.090023 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.089892    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jwf7q\""
Sep 05 07:49:34.093603 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.093581    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f48649c8-7b27q"]
Sep 05 07:49:34.162851 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.162823    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-oauth-config\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.163012 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.162893    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-console-config\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.163012 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.162908    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-service-ca\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.163012 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.162932    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-serving-cert\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.163012 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.162955    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-oauth-serving-cert\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.163012 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.162979    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbdj4\" (UniqueName: \"kubernetes.io/projected/f64b9381-137e-425f-961e-389e4d0b0f61-kube-api-access-wbdj4\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.263985 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.263941    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-console-config\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.264178 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.264009    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-service-ca\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.264178 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.264047    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-serving-cert\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.264178 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.264084    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-oauth-serving-cert\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.264178 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.264108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbdj4\" (UniqueName: \"kubernetes.io/projected/f64b9381-137e-425f-961e-389e4d0b0f61-kube-api-access-wbdj4\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.264178 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.264145    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-oauth-config\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.266503 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.265419    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-oauth-serving-cert\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.266503 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.266064    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-service-ca\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.266503 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.266465    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-console-config\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.268225 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.268181    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-oauth-config\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.269406 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.269386    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-serving-cert\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.278093 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.278048    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbdj4\" (UniqueName: \"kubernetes.io/projected/f64b9381-137e-425f-961e-389e4d0b0f61-kube-api-access-wbdj4\") pod \"console-74f48649c8-7b27q\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") " pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.397861 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.397821    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:34.680423 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.680344    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f48649c8-7b27q"]
Sep 05 07:49:34.683968 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:34.683938    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64b9381_137e_425f_961e_389e4d0b0f61.slice/crio-d89ccc8b0879d1d9c99e19c531935fff188c466ffd49959d14172d20f6c0f5d2 WatchSource:0}: Error finding container d89ccc8b0879d1d9c99e19c531935fff188c466ffd49959d14172d20f6c0f5d2: Status 404 returned error can't find the container with id d89ccc8b0879d1d9c99e19c531935fff188c466ffd49959d14172d20f6c0f5d2
Sep 05 07:49:34.783413 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.783375    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f48649c8-7b27q" event={"ID":"f64b9381-137e-425f-961e-389e4d0b0f61","Type":"ContainerStarted","Data":"d89ccc8b0879d1d9c99e19c531935fff188c466ffd49959d14172d20f6c0f5d2"}
Sep 05 07:49:34.785494 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:34.785454    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ph7k8" event={"ID":"bbfb7b6c-970f-4d09-8eaf-e4944b723741","Type":"ContainerStarted","Data":"4309495c3ece275ace63f7e8da49387a392a48cf71598e430fb6fdb5fdc04912"}
Sep 05 07:49:35.790508 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:35.790423    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vtcdn" event={"ID":"1a0f2752-8a69-451c-a57d-c38242e3350d","Type":"ContainerStarted","Data":"e85c81a17fcaf84f376b8fa8f18a1394861f39c8741c1dc79bd6f34ccc79d5c7"}
Sep 05 07:49:37.727962 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.727931    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg"]
Sep 05 07:49:37.732334 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.732312    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg"
Sep 05 07:49:37.735411 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.735386    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 05 07:49:37.735495 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.735434    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rzs4v\""
Sep 05 07:49:37.740960 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.740939    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg"]
Sep 05 07:49:37.787961 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.787915    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5f5d1728-d25d-41ba-9279-6d688077e828-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-t2tsg\" (UID: \"5f5d1728-d25d-41ba-9279-6d688077e828\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg"
Sep 05 07:49:37.888643 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.888575    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5f5d1728-d25d-41ba-9279-6d688077e828-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-t2tsg\" (UID: \"5f5d1728-d25d-41ba-9279-6d688077e828\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg"
Sep 05 07:49:37.891545 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.891497    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-568f4bdd46-6lxfd"]
Sep 05 07:49:37.891757 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.891709    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5f5d1728-d25d-41ba-9279-6d688077e828-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-t2tsg\" (UID: \"5f5d1728-d25d-41ba-9279-6d688077e828\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg"
Sep 05 07:49:37.895967 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.895945    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:37.904782 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.904758    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 05 07:49:37.907136 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.907112    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-568f4bdd46-6lxfd"]
Sep 05 07:49:37.989442 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.989351    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-serving-cert\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:37.989442 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.989417    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-trusted-ca-bundle\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:37.989684 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.989445    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-oauth-serving-cert\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:37.989684 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.989488    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-console-config\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:37.989684 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.989561    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-oauth-config\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:37.989684 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.989580    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-service-ca\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:37.989684 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:37.989597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2lfb\" (UniqueName: \"kubernetes.io/projected/49b3d61c-e946-48c6-8584-9b8680a3149f-kube-api-access-s2lfb\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.044389 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.044355    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg"
Sep 05 07:49:38.090731 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.090704    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-serving-cert\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.090803 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.090749    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-trusted-ca-bundle\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.090803 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.090771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-oauth-serving-cert\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.090864 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.090799    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-console-config\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.092196 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.091358    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-oauth-config\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.092196 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.091416    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-service-ca\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.092196 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.091450    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2lfb\" (UniqueName: \"kubernetes.io/projected/49b3d61c-e946-48c6-8584-9b8680a3149f-kube-api-access-s2lfb\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.092196 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.091979    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-console-config\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.092471 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.092227    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-trusted-ca-bundle\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.095669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.092932    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-oauth-serving-cert\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.095669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.093136    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-service-ca\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.095669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.093942    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-serving-cert\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.095669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.095163    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-oauth-config\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.101118 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.101100    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2lfb\" (UniqueName: \"kubernetes.io/projected/49b3d61c-e946-48c6-8584-9b8680a3149f-kube-api-access-s2lfb\") pod \"console-568f4bdd46-6lxfd\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") " pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.208280 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.208090    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:38.419287 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.419254    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg"]
Sep 05 07:49:38.422202 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:38.422174    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5d1728_d25d_41ba_9279_6d688077e828.slice/crio-ee67dbcd1cc606a31b61fd545c0cf09587f6bf24be2baa14b425f4635d6ac92e WatchSource:0}: Error finding container ee67dbcd1cc606a31b61fd545c0cf09587f6bf24be2baa14b425f4635d6ac92e: Status 404 returned error can't find the container with id ee67dbcd1cc606a31b61fd545c0cf09587f6bf24be2baa14b425f4635d6ac92e
Sep 05 07:49:38.502364 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.502335    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-568f4bdd46-6lxfd"]
Sep 05 07:49:38.504865 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:38.504839    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b3d61c_e946_48c6_8584_9b8680a3149f.slice/crio-a5a4ce29550592804eeaadaddaeee1ecf514ec00e55df31f2b675bda5121472f WatchSource:0}: Error finding container a5a4ce29550592804eeaadaddaeee1ecf514ec00e55df31f2b675bda5121472f: Status 404 returned error can't find the container with id a5a4ce29550592804eeaadaddaeee1ecf514ec00e55df31f2b675bda5121472f
Sep 05 07:49:38.724033 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.724008    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp7vx"
Sep 05 07:49:38.803049 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.802970    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ksjgr" event={"ID":"735870a0-b9d2-4e05-8e45-7e01af14f5a3","Type":"ContainerStarted","Data":"ccf160aa891be805dc2c82f8fb59bd3683989a557ae0423f66c5002ddd55958d"}
Sep 05 07:49:38.804887 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.804863    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vtcdn" event={"ID":"1a0f2752-8a69-451c-a57d-c38242e3350d","Type":"ContainerStarted","Data":"8c3122e09f0b36240a979f4569814828392c21de8bd7819d4c194d383cf2366a"}
Sep 05 07:49:38.804989 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.804926    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:38.806441 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.806422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f48649c8-7b27q" event={"ID":"f64b9381-137e-425f-961e-389e4d0b0f61","Type":"ContainerStarted","Data":"76fb51544a01ce99f40bec355ceb1ac4a3e7fca15a047a6a49ac5388f776f579"}
Sep 05 07:49:38.808073 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.808049    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568f4bdd46-6lxfd" event={"ID":"49b3d61c-e946-48c6-8584-9b8680a3149f","Type":"ContainerStarted","Data":"f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda"}
Sep 05 07:49:38.808167 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.808089    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568f4bdd46-6lxfd" event={"ID":"49b3d61c-e946-48c6-8584-9b8680a3149f","Type":"ContainerStarted","Data":"a5a4ce29550592804eeaadaddaeee1ecf514ec00e55df31f2b675bda5121472f"}
Sep 05 07:49:38.809174 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.809153    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg" event={"ID":"5f5d1728-d25d-41ba-9279-6d688077e828","Type":"ContainerStarted","Data":"ee67dbcd1cc606a31b61fd545c0cf09587f6bf24be2baa14b425f4635d6ac92e"}
Sep 05 07:49:38.811055 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.811037    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ph7k8" event={"ID":"bbfb7b6c-970f-4d09-8eaf-e4944b723741","Type":"ContainerStarted","Data":"70923956b2475633929295ad762d574fc8a43a2188dd7cafab224e21fe498b0d"}
Sep 05 07:49:38.821170 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.821131    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ksjgr" podStartSLOduration=1.3347108859999999 podStartE2EDuration="5.821119161s" totalImagesPullingTime="4.486408275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:38.820213548 +0000 UTC m=+59.765535352" watchObservedRunningTime="2026-09-05 07:49:38.821119161 +0000 UTC m=+59.766440962"
Sep 05 07:49:38.840419 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.840371    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-568f4bdd46-6lxfd" podStartSLOduration=1.840358539 podStartE2EDuration="1.840358539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:38.839772102 +0000 UTC m=+59.785093908" watchObservedRunningTime="2026-09-05 07:49:38.840358539 +0000 UTC m=+59.785680392"
Sep 05 07:49:38.861057 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.861013    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ph7k8" podStartSLOduration=2.456168037 podStartE2EDuration="6.860999416s" totalImagesPullingTime="4.404831379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:49:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:38.860500757 +0000 UTC m=+59.805822561" watchObservedRunningTime="2026-09-05 07:49:38.860999416 +0000 UTC m=+59.806321220"
Sep 05 07:49:38.880224 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:38.880181    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74f48649c8-7b27q" podStartSLOduration=1.419366446 podStartE2EDuration="4.880167147s" totalImagesPullingTime="3.460800701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:38.879753756 +0000 UTC m=+59.825075562" watchObservedRunningTime="2026-09-05 07:49:38.880167147 +0000 UTC m=+59.825488951"
Sep 05 07:49:39.816234 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:39.816144    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg" event={"ID":"5f5d1728-d25d-41ba-9279-6d688077e828","Type":"ContainerStarted","Data":"b8286e6ee924b480b05907519f8570063aa241ac13f6f67156a7e24a8b5d1258"}
Sep 05 07:49:39.833811 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:39.833766    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vtcdn" podStartSLOduration=6.062820243 podStartE2EDuration="7.833752692s" totalImagesPullingTime="1.770932449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:38.896899555 +0000 UTC m=+59.842221367" watchObservedRunningTime="2026-09-05 07:49:39.833752692 +0000 UTC m=+60.779074496"
Sep 05 07:49:39.834106 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:39.834086    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg" podStartSLOduration=1.7669292730000001 podStartE2EDuration="2.834079827s" totalImagesPullingTime="1.067150554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:39.833502891 +0000 UTC m=+60.778824696" watchObservedRunningTime="2026-09-05 07:49:39.834079827 +0000 UTC m=+60.779401632"
Sep 05 07:49:40.819175 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:40.819140    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg"
Sep 05 07:49:40.823865 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:40.823839    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-t2tsg"
Sep 05 07:49:41.790717 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.790687    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"]
Sep 05 07:49:41.793828 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.793814    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.797960 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.797933    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 05 07:49:41.798107 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.797963    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 05 07:49:41.798107 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.797996    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-9cmm8\""
Sep 05 07:49:41.798107 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.797945    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 05 07:49:41.798107 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.798029    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 05 07:49:41.798107 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.798087    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 05 07:49:41.801555 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.801501    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"]
Sep 05 07:49:41.824303 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.824281    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0764a774-f924-49d5-ac9a-c006a2aeae64-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.824607 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.824311    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz992\" (UniqueName: \"kubernetes.io/projected/0764a774-f924-49d5-ac9a-c006a2aeae64-kube-api-access-nz992\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.824607 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.824336    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0764a774-f924-49d5-ac9a-c006a2aeae64-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.824607 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.824369    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0764a774-f924-49d5-ac9a-c006a2aeae64-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.924746 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.924719    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0764a774-f924-49d5-ac9a-c006a2aeae64-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.924839 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.924763    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz992\" (UniqueName: \"kubernetes.io/projected/0764a774-f924-49d5-ac9a-c006a2aeae64-kube-api-access-nz992\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.924896 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.924870    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0764a774-f924-49d5-ac9a-c006a2aeae64-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.924953 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.924915    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0764a774-f924-49d5-ac9a-c006a2aeae64-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.925037 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:41.925016    2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 05 07:49:41.925109 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:41.925097    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0764a774-f924-49d5-ac9a-c006a2aeae64-prometheus-operator-tls podName:0764a774-f924-49d5-ac9a-c006a2aeae64 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:42.425078013 +0000 UTC m=+63.370399806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0764a774-f924-49d5-ac9a-c006a2aeae64-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-jpf6c" (UID: "0764a774-f924-49d5-ac9a-c006a2aeae64") : secret "prometheus-operator-tls" not found
Sep 05 07:49:41.925458 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.925441    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0764a774-f924-49d5-ac9a-c006a2aeae64-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.927212 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.927195    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0764a774-f924-49d5-ac9a-c006a2aeae64-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:41.933572 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:41.933552    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz992\" (UniqueName: \"kubernetes.io/projected/0764a774-f924-49d5-ac9a-c006a2aeae64-kube-api-access-nz992\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:42.429224 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:42.429189    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0764a774-f924-49d5-ac9a-c006a2aeae64-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:42.431780 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:42.431755    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0764a774-f924-49d5-ac9a-c006a2aeae64-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-jpf6c\" (UID: \"0764a774-f924-49d5-ac9a-c006a2aeae64\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:42.702657 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:42.702593    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"
Sep 05 07:49:42.995885 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:42.995861    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-jpf6c"]
Sep 05 07:49:42.997886 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:42.997864    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0764a774_f924_49d5_ac9a_c006a2aeae64.slice/crio-960c632c4099ccf6a68c9f47a66872223f40e414fa79f2ec57a7d8338d70a2ae WatchSource:0}: Error finding container 960c632c4099ccf6a68c9f47a66872223f40e414fa79f2ec57a7d8338d70a2ae: Status 404 returned error can't find the container with id 960c632c4099ccf6a68c9f47a66872223f40e414fa79f2ec57a7d8338d70a2ae
Sep 05 07:49:43.829370 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:43.829326    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c" event={"ID":"0764a774-f924-49d5-ac9a-c006a2aeae64","Type":"ContainerStarted","Data":"960c632c4099ccf6a68c9f47a66872223f40e414fa79f2ec57a7d8338d70a2ae"}
Sep 05 07:49:44.398787 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:44.398758    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:44.399074 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:44.398819    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:44.404070 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:44.404044    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:44.835019 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:44.834981    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c" event={"ID":"0764a774-f924-49d5-ac9a-c006a2aeae64","Type":"ContainerStarted","Data":"5d5a3129c487b467a50b525175270795f81baef83a98f659c99e8f65662ad0f7"}
Sep 05 07:49:44.835182 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:44.835043    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c" event={"ID":"0764a774-f924-49d5-ac9a-c006a2aeae64","Type":"ContainerStarted","Data":"cc590f29196cd00d1fd76b87f330da7392b551f8228b7942937b815cdd4d5e55"}
Sep 05 07:49:44.839129 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:44.839104    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:49:44.854171 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:44.854116    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-jpf6c" podStartSLOduration=2.534501601 podStartE2EDuration="3.854098875s" totalImagesPullingTime="1.319597274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:44.851464816 +0000 UTC m=+65.796786618" watchObservedRunningTime="2026-09-05 07:49:44.854098875 +0000 UTC m=+65.799420683"
Sep 05 07:49:45.252945 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.252849    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7t5\" (UniqueName: \"kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5\") pod \"network-check-target-7njm4\" (UID: \"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc\") " pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:45.252945 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.252905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:45.255910 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.255889    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 05 07:49:45.256052 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.255996    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 05 07:49:45.265356 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.265325    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6264509b-4382-45b2-a6f0-d8af26dd57f1-metrics-certs\") pod \"network-metrics-daemon-bcxfm\" (UID: \"6264509b-4382-45b2-a6f0-d8af26dd57f1\") " pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:45.265456 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.265397    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 05 07:49:45.275803 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.275784    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z7t5\" (UniqueName: \"kubernetes.io/projected/c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc-kube-api-access-9z7t5\") pod \"network-check-target-7njm4\" (UID: \"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc\") " pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:45.277573 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.277559    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hp5x9\""
Sep 05 07:49:45.284214 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.284198    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g9xp6\""
Sep 05 07:49:45.285243 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.285229    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bcxfm"
Sep 05 07:49:45.292289 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.292271    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:45.354381 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.354088    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:45.357069 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.356978    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 05 07:49:45.367408 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.367385    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a-original-pull-secret\") pod \"global-pull-secret-syncer-6nw26\" (UID: \"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a\") " pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:45.555014 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.554803    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bcxfm"]
Sep 05 07:49:45.557323 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:45.557294    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6264509b_4382_45b2_a6f0_d8af26dd57f1.slice/crio-45545f141f3dae579e1529dbb9377271e6539c9dc556d51571c397285c97b051 WatchSource:0}: Error finding container 45545f141f3dae579e1529dbb9377271e6539c9dc556d51571c397285c97b051: Status 404 returned error can't find the container with id 45545f141f3dae579e1529dbb9377271e6539c9dc556d51571c397285c97b051
Sep 05 07:49:45.573677 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.573657    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7njm4"]
Sep 05 07:49:45.575700 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:45.575669    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc49e45cb_ca6e_4d07_b1ec_e6e175ebe6bc.slice/crio-e2844894e7a759983ab6eebd71578ade1094145f3b28ffe62142cbf6e30bd9db WatchSource:0}: Error finding container e2844894e7a759983ab6eebd71578ade1094145f3b28ffe62142cbf6e30bd9db: Status 404 returned error can't find the container with id e2844894e7a759983ab6eebd71578ade1094145f3b28ffe62142cbf6e30bd9db
Sep 05 07:49:45.591804 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.591786    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6nw26"
Sep 05 07:49:45.819125 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.819100    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vtcdn"
Sep 05 07:49:45.839577 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.839554    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7njm4" event={"ID":"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc","Type":"ContainerStarted","Data":"e2844894e7a759983ab6eebd71578ade1094145f3b28ffe62142cbf6e30bd9db"}
Sep 05 07:49:45.841187 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.841151    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bcxfm" event={"ID":"6264509b-4382-45b2-a6f0-d8af26dd57f1","Type":"ContainerStarted","Data":"45545f141f3dae579e1529dbb9377271e6539c9dc556d51571c397285c97b051"}
Sep 05 07:49:45.868415 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:45.868287    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6nw26"]
Sep 05 07:49:45.870782 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:45.870753    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bb3f562_7acc_4fcd_8f35_b4b8e18e2a5a.slice/crio-8e3d9c2e5436d6a55b98de394719e0d2613fca69fab9278a0080b808f5369ffa WatchSource:0}: Error finding container 8e3d9c2e5436d6a55b98de394719e0d2613fca69fab9278a0080b808f5369ffa: Status 404 returned error can't find the container with id 8e3d9c2e5436d6a55b98de394719e0d2613fca69fab9278a0080b808f5369ffa
Sep 05 07:49:46.848033 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:46.847993    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bcxfm" event={"ID":"6264509b-4382-45b2-a6f0-d8af26dd57f1","Type":"ContainerStarted","Data":"594d1ebc140a779c65b44ce0de081b589c1db830545f5a82f403f0244bf49f8f"}
Sep 05 07:49:46.848483 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:46.848051    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bcxfm" event={"ID":"6264509b-4382-45b2-a6f0-d8af26dd57f1","Type":"ContainerStarted","Data":"cf3738995c936fa6a93d2cbe4e8ff6bdc2f09f46a8d8de87c27fef11be7ea586"}
Sep 05 07:49:46.850542 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:46.850497    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6nw26" event={"ID":"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a","Type":"ContainerStarted","Data":"8e3d9c2e5436d6a55b98de394719e0d2613fca69fab9278a0080b808f5369ffa"}
Sep 05 07:49:47.253031 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.252893    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bcxfm" podStartSLOduration=67.269535076 podStartE2EDuration="1m8.252871715s" totalImagesPullingTime="983.336639ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:46.871617762 +0000 UTC m=+67.816939594" watchObservedRunningTime="2026-09-05 07:49:47.252871715 +0000 UTC m=+68.198193523"
Sep 05 07:49:47.253196 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.253128    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"]
Sep 05 07:49:47.256542 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.256510    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.259696 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.259674    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 05 07:49:47.259854 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.259842    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 05 07:49:47.260188 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.260162    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-schlj\""
Sep 05 07:49:47.275557 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.275513    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"]
Sep 05 07:49:47.290410 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.290388    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"]
Sep 05 07:49:47.294895 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.294869    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.299426 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.299402    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qddbv\""
Sep 05 07:49:47.299641 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.299624    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 05 07:49:47.299781 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.299761    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 05 07:49:47.301072 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.301053    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 05 07:49:47.323029 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.317567    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"]
Sep 05 07:49:47.323029 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.319187    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-w524v"]
Sep 05 07:49:47.324627 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.324581    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.327681 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.327666    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9tz6f\""
Sep 05 07:49:47.328186 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.328148    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 05 07:49:47.328774 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.328733    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 05 07:49:47.331312 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.331293    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 05 07:49:47.372797 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.372762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.372797 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.372794    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-accelerators-collector-config\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.372995 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.372826    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.372995 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.372857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-textfile\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.373093 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373049    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.373211 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373175    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/de5cc2c8-7bd9-4706-8788-8b8479a7087a-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.373325 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373221    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.373325 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373249    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bab5ab31-4965-420e-a2e2-7c53dcd98c85-root\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.373325 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373280    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.373325 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373308    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.373494 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373340    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de5cc2c8-7bd9-4706-8788-8b8479a7087a-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.373494 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373364    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg4rv\" (UniqueName: \"kubernetes.io/projected/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-kube-api-access-mg4rv\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.373494 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bab5ab31-4965-420e-a2e2-7c53dcd98c85-metrics-client-ca\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.373494 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373418    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bab5ab31-4965-420e-a2e2-7c53dcd98c85-sys\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.373494 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373487    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp4hc\" (UniqueName: \"kubernetes.io/projected/bab5ab31-4965-420e-a2e2-7c53dcd98c85-kube-api-access-hp4hc\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.373850 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373580    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-tls\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.373850 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373607    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.373850 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373636    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mht7x\" (UniqueName: \"kubernetes.io/projected/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-api-access-mht7x\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.373850 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.373684    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-wtmp\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.474338 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.474297    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.474521 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.474346    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-accelerators-collector-config\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.474849 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.474818    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.474943 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.474881    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-textfile\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.474943 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.474916    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.475041 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.474944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/de5cc2c8-7bd9-4706-8788-8b8479a7087a-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.475041 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.474951    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-accelerators-collector-config\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.475041 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.474968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.475041 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.474985    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bab5ab31-4965-420e-a2e2-7c53dcd98c85-root\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.475041 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475010    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.475041 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.475323 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475050    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de5cc2c8-7bd9-4706-8788-8b8479a7087a-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.475323 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg4rv\" (UniqueName: \"kubernetes.io/projected/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-kube-api-access-mg4rv\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.475323 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475094    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bab5ab31-4965-420e-a2e2-7c53dcd98c85-metrics-client-ca\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.475323 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475125    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bab5ab31-4965-420e-a2e2-7c53dcd98c85-sys\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.475323 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:47.475135    2569 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 05 07:49:47.475323 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp4hc\" (UniqueName: \"kubernetes.io/projected/bab5ab31-4965-420e-a2e2-7c53dcd98c85-kube-api-access-hp4hc\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.475323 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:47.475197    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-openshift-state-metrics-tls podName:8329852e-bd3d-453d-b2ee-ae97d1b6f8cd nodeName:}" failed. No retries permitted until 2026-09-05 07:49:47.975177396 +0000 UTC m=+68.920499182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-67csw" (UID: "8329852e-bd3d-453d-b2ee-ae97d1b6f8cd") : secret "openshift-state-metrics-tls" not found
Sep 05 07:49:47.475323 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-tls\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.475323 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.475323 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475317    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mht7x\" (UniqueName: \"kubernetes.io/projected/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-api-access-mht7x\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.475808 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475358    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-wtmp\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.475808 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475553    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-wtmp\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.475808 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475738    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/de5cc2c8-7bd9-4706-8788-8b8479a7087a-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.475954 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.475811    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-textfile\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.476871 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.476842    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.477304 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.477063    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de5cc2c8-7bd9-4706-8788-8b8479a7087a-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.477304 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.477233    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bab5ab31-4965-420e-a2e2-7c53dcd98c85-metrics-client-ca\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.477304 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.477291    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bab5ab31-4965-420e-a2e2-7c53dcd98c85-sys\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.477489 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.477315    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bab5ab31-4965-420e-a2e2-7c53dcd98c85-root\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.477489 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.477383    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.478028 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.477989    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.479822 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.479779    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.480362 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.480340    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bab5ab31-4965-420e-a2e2-7c53dcd98c85-node-exporter-tls\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.481455 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.481425    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.483192 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.483168    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.488428 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.488406    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp4hc\" (UniqueName: \"kubernetes.io/projected/bab5ab31-4965-420e-a2e2-7c53dcd98c85-kube-api-access-hp4hc\") pod \"node-exporter-w524v\" (UID: \"bab5ab31-4965-420e-a2e2-7c53dcd98c85\") " pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.488814 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.488789    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg4rv\" (UniqueName: \"kubernetes.io/projected/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-kube-api-access-mg4rv\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.489088 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.489063    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mht7x\" (UniqueName: \"kubernetes.io/projected/de5cc2c8-7bd9-4706-8788-8b8479a7087a-kube-api-access-mht7x\") pod \"kube-state-metrics-74cb57df7-hhgqh\" (UID: \"de5cc2c8-7bd9-4706-8788-8b8479a7087a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.606650 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.606618    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"
Sep 05 07:49:47.638133 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.638106    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w524v"
Sep 05 07:49:47.980466 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.980372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:47.991071 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:47.991042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8329852e-bd3d-453d-b2ee-ae97d1b6f8cd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-67csw\" (UID: \"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:48.169549 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.169494    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"
Sep 05 07:49:48.209665 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.209226    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:48.209665 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.209474    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:48.215094 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.215072    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:48.262711 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.262634    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 07:49:48.266423 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.266398    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.270717 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.270696    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 05 07:49:48.270919 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.270903    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 05 07:49:48.272003 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.271000    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 05 07:49:48.272003 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.271118    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 05 07:49:48.272003 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.271227    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 05 07:49:48.272003 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.271406    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 05 07:49:48.272003 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.271422    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 05 07:49:48.272003 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.271569    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 05 07:49:48.272003 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.271615    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 05 07:49:48.272003 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.271797    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s48qk\""
Sep 05 07:49:48.285151 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.285067    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 07:49:48.385015 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.384813    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-config-volume\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385015 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.384856    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385015 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.384884    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20073340-5926-4f77-b1ff-f70558d61bb2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385015 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.384931    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385015 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.384971    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385344 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.385067    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385344 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.385100    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwsfv\" (UniqueName: \"kubernetes.io/projected/20073340-5926-4f77-b1ff-f70558d61bb2-kube-api-access-cwsfv\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385344 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.385118    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20073340-5926-4f77-b1ff-f70558d61bb2-config-out\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385344 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.385148    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-web-config\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385344 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.385171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20073340-5926-4f77-b1ff-f70558d61bb2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385344 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.385196    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/20073340-5926-4f77-b1ff-f70558d61bb2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385344 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.385213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20073340-5926-4f77-b1ff-f70558d61bb2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.385344 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.385260    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.485897 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.485859    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.485897 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.485910    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486122 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.485978    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486122 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.486008    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwsfv\" (UniqueName: \"kubernetes.io/projected/20073340-5926-4f77-b1ff-f70558d61bb2-kube-api-access-cwsfv\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486122 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.486033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20073340-5926-4f77-b1ff-f70558d61bb2-config-out\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486122 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.486065    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-web-config\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486122 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.486094    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20073340-5926-4f77-b1ff-f70558d61bb2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486372 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.486131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/20073340-5926-4f77-b1ff-f70558d61bb2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486372 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.486153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20073340-5926-4f77-b1ff-f70558d61bb2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486372 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.486189    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486372 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.486218    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-config-volume\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486372 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.486242    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486372 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.486267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20073340-5926-4f77-b1ff-f70558d61bb2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.486741 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:48.486720    2569 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 05 07:49:48.486800 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:48.486789    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-main-tls podName:20073340-5926-4f77-b1ff-f70558d61bb2 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:48.986769254 +0000 UTC m=+69.932091051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "20073340-5926-4f77-b1ff-f70558d61bb2") : secret "alertmanager-main-tls" not found
Sep 05 07:49:48.487568 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.487331    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/20073340-5926-4f77-b1ff-f70558d61bb2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.487568 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.487493    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20073340-5926-4f77-b1ff-f70558d61bb2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.491553 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.488217    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20073340-5926-4f77-b1ff-f70558d61bb2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.491553 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.490250    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20073340-5926-4f77-b1ff-f70558d61bb2-config-out\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.491553 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.490624    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20073340-5926-4f77-b1ff-f70558d61bb2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.491553 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.490672    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-web-config\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.491553 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.491148    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.491553 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.491155    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.492007 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.491941    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.492289 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.492265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-config-volume\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.492750 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.492728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.503133 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.503089    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwsfv\" (UniqueName: \"kubernetes.io/projected/20073340-5926-4f77-b1ff-f70558d61bb2-kube-api-access-cwsfv\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.863942 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.863912    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:49:48.911164 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.911134    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74f48649c8-7b27q"]
Sep 05 07:49:48.995204 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.995167    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:48.998042 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:48.998021    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/20073340-5926-4f77-b1ff-f70558d61bb2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"20073340-5926-4f77-b1ff-f70558d61bb2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:49.179587 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:49.179505    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 05 07:49:50.189266 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.189238    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8664b866fc-qlcph"]
Sep 05 07:49:50.193761 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.193745    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.196985 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.196967    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 05 07:49:50.197302 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.197278    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 05 07:49:50.197411 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.197342    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 05 07:49:50.197411 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.197370    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-agq88tuj7amqf\""
Sep 05 07:49:50.197411 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.197348    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-s624b\""
Sep 05 07:49:50.198341 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.198325    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 05 07:49:50.198469 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.198372    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 05 07:49:50.202690 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.202669    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8664b866fc-qlcph"]
Sep 05 07:49:50.308008 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.307568    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.308008 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.307616    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-tls\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.308008 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.307656    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.308008 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.307722    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.308008 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.307750    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7277c30-a1dd-4fbf-b05c-421072627753-metrics-client-ca\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.308008 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.307776    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvwr9\" (UniqueName: \"kubernetes.io/projected/a7277c30-a1dd-4fbf-b05c-421072627753-kube-api-access-cvwr9\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.308008 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.307832    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-grpc-tls\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.308008 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.307867    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.408774 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.408746    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.408856 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.408792    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.408856 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.408811    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-tls\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.408856 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.408834    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.408969 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.408875    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.408969 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.408897    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7277c30-a1dd-4fbf-b05c-421072627753-metrics-client-ca\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.408969 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.408920    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvwr9\" (UniqueName: \"kubernetes.io/projected/a7277c30-a1dd-4fbf-b05c-421072627753-kube-api-access-cvwr9\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.409131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.409107    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-grpc-tls\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.410002 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.409936    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7277c30-a1dd-4fbf-b05c-421072627753-metrics-client-ca\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.412067 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.411705    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-grpc-tls\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.412067 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.412011    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.412067 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.412019    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.412300 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.412161    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.412300 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.412246    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-tls\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.413477 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.413460    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a7277c30-a1dd-4fbf-b05c-421072627753-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.417230 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.417207    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvwr9\" (UniqueName: \"kubernetes.io/projected/a7277c30-a1dd-4fbf-b05c-421072627753-kube-api-access-cvwr9\") pod \"thanos-querier-8664b866fc-qlcph\" (UID: \"a7277c30-a1dd-4fbf-b05c-421072627753\") " pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.473111 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.473087    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh"]
Sep 05 07:49:50.475437 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:50.475407    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5cc2c8_7bd9_4706_8788_8b8479a7087a.slice/crio-5d2279a2c36acf86aa5743852f59a5e6a2a65c6ae5769ac1324dca4a69026cf8 WatchSource:0}: Error finding container 5d2279a2c36acf86aa5743852f59a5e6a2a65c6ae5769ac1324dca4a69026cf8: Status 404 returned error can't find the container with id 5d2279a2c36acf86aa5743852f59a5e6a2a65c6ae5769ac1324dca4a69026cf8
Sep 05 07:49:50.503790 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.503764    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:50.527469 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.527383    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-67csw"]
Sep 05 07:49:50.529043 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:50.529016    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8329852e_bd3d_453d_b2ee_ae97d1b6f8cd.slice/crio-46040b1c558fc71c12b66ecd20e24751a9dfdf73536d42efddaf4c9450a4cc5b WatchSource:0}: Error finding container 46040b1c558fc71c12b66ecd20e24751a9dfdf73536d42efddaf4c9450a4cc5b: Status 404 returned error can't find the container with id 46040b1c558fc71c12b66ecd20e24751a9dfdf73536d42efddaf4c9450a4cc5b
Sep 05 07:49:50.571161 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.571065    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 07:49:50.574575 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:50.574518    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20073340_5926_4f77_b1ff_f70558d61bb2.slice/crio-52a50de7b12cfc260a5ce6eb10aa93dbcd748f07394e5bce9b770aea804bb113 WatchSource:0}: Error finding container 52a50de7b12cfc260a5ce6eb10aa93dbcd748f07394e5bce9b770aea804bb113: Status 404 returned error can't find the container with id 52a50de7b12cfc260a5ce6eb10aa93dbcd748f07394e5bce9b770aea804bb113
Sep 05 07:49:50.791285 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.791194    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8664b866fc-qlcph"]
Sep 05 07:49:50.794487 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:50.794453    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7277c30_a1dd_4fbf_b05c_421072627753.slice/crio-7d495be34df67fba701a70ee8383aabc87f165e422d4406a974c42af588cbb98 WatchSource:0}: Error finding container 7d495be34df67fba701a70ee8383aabc87f165e422d4406a974c42af588cbb98: Status 404 returned error can't find the container with id 7d495be34df67fba701a70ee8383aabc87f165e422d4406a974c42af588cbb98
Sep 05 07:49:50.869636 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.869564    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6nw26" event={"ID":"3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a","Type":"ContainerStarted","Data":"08137f45388d2acae4686c83292d93e42ffb522aaf9fcda9cab567210a192a83"}
Sep 05 07:49:50.871223 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.871182    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph" event={"ID":"a7277c30-a1dd-4fbf-b05c-421072627753","Type":"ContainerStarted","Data":"7d495be34df67fba701a70ee8383aabc87f165e422d4406a974c42af588cbb98"}
Sep 05 07:49:50.872590 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.872563    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20073340-5926-4f77-b1ff-f70558d61bb2","Type":"ContainerStarted","Data":"52a50de7b12cfc260a5ce6eb10aa93dbcd748f07394e5bce9b770aea804bb113"}
Sep 05 07:49:50.874654 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.874510    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw" event={"ID":"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd","Type":"ContainerStarted","Data":"65d827b6bf5346bd071ecfe09f046e476842164fca75e01deb9881567adccb38"}
Sep 05 07:49:50.874654 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.874582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw" event={"ID":"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd","Type":"ContainerStarted","Data":"fe13ca9b9834200267ba2f86601097a6efd562731eec2bc70481376252ade00b"}
Sep 05 07:49:50.874654 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.874610    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw" event={"ID":"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd","Type":"ContainerStarted","Data":"46040b1c558fc71c12b66ecd20e24751a9dfdf73536d42efddaf4c9450a4cc5b"}
Sep 05 07:49:50.875866 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.875841    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh" event={"ID":"de5cc2c8-7bd9-4706-8788-8b8479a7087a","Type":"ContainerStarted","Data":"5d2279a2c36acf86aa5743852f59a5e6a2a65c6ae5769ac1324dca4a69026cf8"}
Sep 05 07:49:50.877743 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.877707    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7njm4" event={"ID":"c49e45cb-ca6e-4d07-b1ec-e6e175ebe6bc","Type":"ContainerStarted","Data":"7f347d91ea7a119de272a3dcff23411fd72b234db6ebaed93660aa4f615bffde"}
Sep 05 07:49:50.877898 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.877880    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:49:50.879075 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.879046    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w524v" event={"ID":"bab5ab31-4965-420e-a2e2-7c53dcd98c85","Type":"ContainerStarted","Data":"5f14fb9253a24e8779aba42ae880f2eb4044daeb5e9e066798b385010c255704"}
Sep 05 07:49:50.898410 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.898367    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6nw26" podStartSLOduration=67.512407581 podStartE2EDuration="1m11.89835526s" totalImagesPullingTime="4.385947679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:50.898110461 +0000 UTC m=+71.843432266" watchObservedRunningTime="2026-09-05 07:49:50.89835526 +0000 UTC m=+71.843677059"
Sep 05 07:49:50.918445 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:50.918406    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7njm4" podStartSLOduration=67.248270834 podStartE2EDuration="1m11.918389009s" totalImagesPullingTime="4.670118175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:50.917572924 +0000 UTC m=+71.862894730" watchObservedRunningTime="2026-09-05 07:49:50.918389009 +0000 UTC m=+71.863710813"
Sep 05 07:49:51.610697 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.610663    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-769d67686c-7gbxm"]
Sep 05 07:49:51.614702 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.613796    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.617162 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.617138    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 05 07:49:51.617420 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.617401    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 05 07:49:51.618082 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.617610    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-67l4uphbqgafa\""
Sep 05 07:49:51.618398 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.618376    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 05 07:49:51.618398 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.618389    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-trmhx\""
Sep 05 07:49:51.619407 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.619386    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 05 07:49:51.624487 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.624457    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-769d67686c-7gbxm"]
Sep 05 07:49:51.720134 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.720101    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkvrf\" (UniqueName: \"kubernetes.io/projected/507dfce2-0001-4920-af9e-21af592e5a3c-kube-api-access-dkvrf\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.720305 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.720158    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/507dfce2-0001-4920-af9e-21af592e5a3c-secret-metrics-server-tls\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.720305 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.720242    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/507dfce2-0001-4920-af9e-21af592e5a3c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.720305 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.720294    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/507dfce2-0001-4920-af9e-21af592e5a3c-audit-log\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.720484 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.720325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/507dfce2-0001-4920-af9e-21af592e5a3c-client-ca-bundle\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.720484 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.720395    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/507dfce2-0001-4920-af9e-21af592e5a3c-secret-metrics-server-client-certs\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.720484 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.720423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/507dfce2-0001-4920-af9e-21af592e5a3c-metrics-server-audit-profiles\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.821654 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.821616    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/507dfce2-0001-4920-af9e-21af592e5a3c-secret-metrics-server-tls\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.821810 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.821677    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/507dfce2-0001-4920-af9e-21af592e5a3c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.821810 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.821711    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/507dfce2-0001-4920-af9e-21af592e5a3c-audit-log\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.821810 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.821735    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/507dfce2-0001-4920-af9e-21af592e5a3c-client-ca-bundle\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.822004 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.821820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/507dfce2-0001-4920-af9e-21af592e5a3c-secret-metrics-server-client-certs\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.822004 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.821856    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/507dfce2-0001-4920-af9e-21af592e5a3c-metrics-server-audit-profiles\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.822004 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.821912    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkvrf\" (UniqueName: \"kubernetes.io/projected/507dfce2-0001-4920-af9e-21af592e5a3c-kube-api-access-dkvrf\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.822575 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.822483    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/507dfce2-0001-4920-af9e-21af592e5a3c-audit-log\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.822703 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.822578    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/507dfce2-0001-4920-af9e-21af592e5a3c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.823219 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.823185    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/507dfce2-0001-4920-af9e-21af592e5a3c-metrics-server-audit-profiles\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.824677 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.824635    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/507dfce2-0001-4920-af9e-21af592e5a3c-secret-metrics-server-tls\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.825236 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.825195    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/507dfce2-0001-4920-af9e-21af592e5a3c-client-ca-bundle\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.827418 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.827395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/507dfce2-0001-4920-af9e-21af592e5a3c-secret-metrics-server-client-certs\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.831830 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.831807    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkvrf\" (UniqueName: \"kubernetes.io/projected/507dfce2-0001-4920-af9e-21af592e5a3c-kube-api-access-dkvrf\") pod \"metrics-server-769d67686c-7gbxm\" (UID: \"507dfce2-0001-4920-af9e-21af592e5a3c\") " pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.887968 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.887897    2569 generic.go:356] "Generic (PLEG): container finished" podID="bab5ab31-4965-420e-a2e2-7c53dcd98c85" containerID="1cd7181e0d01d1660bfeb1d64c5d76a90b1cfad4da21d90b060321fd88b6ffbf" exitCode=0
Sep 05 07:49:51.888216 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.888178    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w524v" event={"ID":"bab5ab31-4965-420e-a2e2-7c53dcd98c85","Type":"ContainerDied","Data":"1cd7181e0d01d1660bfeb1d64c5d76a90b1cfad4da21d90b060321fd88b6ffbf"}
Sep 05 07:49:51.927595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.927568    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:49:51.969017 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.968983    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"]
Sep 05 07:49:51.972826 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.972481    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:51.978069 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.978045    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-s2v9m\""
Sep 05 07:49:51.978298 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.978278    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 05 07:49:51.999930 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:51.999904    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"]
Sep 05 07:49:52.022881 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.022848    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db7e756b-443f-469c-880a-8215b6ac72e3-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-f2nwv\" (UID: \"db7e756b-443f-469c-880a-8215b6ac72e3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:52.023017 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.022889    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scvvv\" (UniqueName: \"kubernetes.io/projected/db7e756b-443f-469c-880a-8215b6ac72e3-kube-api-access-scvvv\") pod \"monitoring-plugin-85bbbbbf4-f2nwv\" (UID: \"db7e756b-443f-469c-880a-8215b6ac72e3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:52.123621 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.123588    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db7e756b-443f-469c-880a-8215b6ac72e3-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-f2nwv\" (UID: \"db7e756b-443f-469c-880a-8215b6ac72e3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:52.123621 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.123623    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scvvv\" (UniqueName: \"kubernetes.io/projected/db7e756b-443f-469c-880a-8215b6ac72e3-kube-api-access-scvvv\") pod \"monitoring-plugin-85bbbbbf4-f2nwv\" (UID: \"db7e756b-443f-469c-880a-8215b6ac72e3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:52.123788 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:52.123754    2569 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 05 07:49:52.123843 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:49:52.123832    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db7e756b-443f-469c-880a-8215b6ac72e3-monitoring-plugin-cert podName:db7e756b-443f-469c-880a-8215b6ac72e3 nodeName:}" failed. No retries permitted until 2026-09-05 07:49:52.623809837 +0000 UTC m=+73.569131624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/db7e756b-443f-469c-880a-8215b6ac72e3-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-f2nwv" (UID: "db7e756b-443f-469c-880a-8215b6ac72e3") : secret "monitoring-plugin-cert" not found
Sep 05 07:49:52.135204 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.135178    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scvvv\" (UniqueName: \"kubernetes.io/projected/db7e756b-443f-469c-880a-8215b6ac72e3-kube-api-access-scvvv\") pod \"monitoring-plugin-85bbbbbf4-f2nwv\" (UID: \"db7e756b-443f-469c-880a-8215b6ac72e3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:52.454238 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.453617    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-596f58f6d6-k92n5"]
Sep 05 07:49:52.457319 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.457296    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.460150 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.460093    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 05 07:49:52.460265 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.460122    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 05 07:49:52.460433 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.460402    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 05 07:49:52.460629 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.460612    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 05 07:49:52.460725 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.460708    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 05 07:49:52.460788 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.460738    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gnwqh\""
Sep 05 07:49:52.470797 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.470771    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-596f58f6d6-k92n5"]
Sep 05 07:49:52.473754 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.473727    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 05 07:49:52.526595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.526564    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-secret-telemeter-client\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.526741 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.526610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf03b4c-c779-49d0-be96-54963187bbb6-serving-certs-ca-bundle\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.526741 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.526634    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hvtd\" (UniqueName: \"kubernetes.io/projected/9cf03b4c-c779-49d0-be96-54963187bbb6-kube-api-access-8hvtd\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.526853 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.526767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-telemeter-client-tls\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.526853 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.526798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.526853 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.526815    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cf03b4c-c779-49d0-be96-54963187bbb6-metrics-client-ca\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.526853 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.526841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf03b4c-c779-49d0-be96-54963187bbb6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.527001 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.526887    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-federate-client-tls\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.627823 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.627787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-telemeter-client-tls\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.628276 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.627835    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.628276 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.627965    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cf03b4c-c779-49d0-be96-54963187bbb6-metrics-client-ca\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.628276 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.628011    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf03b4c-c779-49d0-be96-54963187bbb6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.628276 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.628056    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-federate-client-tls\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.628276 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.628120    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-secret-telemeter-client\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.628276 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.628167    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf03b4c-c779-49d0-be96-54963187bbb6-serving-certs-ca-bundle\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.628276 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.628194    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hvtd\" (UniqueName: \"kubernetes.io/projected/9cf03b4c-c779-49d0-be96-54963187bbb6-kube-api-access-8hvtd\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.628276 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.628247    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db7e756b-443f-469c-880a-8215b6ac72e3-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-f2nwv\" (UID: \"db7e756b-443f-469c-880a-8215b6ac72e3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:52.629296 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.629266    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cf03b4c-c779-49d0-be96-54963187bbb6-metrics-client-ca\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.629844 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.629821    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf03b4c-c779-49d0-be96-54963187bbb6-serving-certs-ca-bundle\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.630884 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.630856    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf03b4c-c779-49d0-be96-54963187bbb6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.630884 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.630876    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-telemeter-client-tls\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.631670 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.631474    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.631742 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.631669    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-federate-client-tls\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.632329 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.632310    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9cf03b4c-c779-49d0-be96-54963187bbb6-secret-telemeter-client\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.633116 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.633100    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db7e756b-443f-469c-880a-8215b6ac72e3-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-f2nwv\" (UID: \"db7e756b-443f-469c-880a-8215b6ac72e3\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:52.640989 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.640967    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hvtd\" (UniqueName: \"kubernetes.io/projected/9cf03b4c-c779-49d0-be96-54963187bbb6-kube-api-access-8hvtd\") pod \"telemeter-client-596f58f6d6-k92n5\" (UID: \"9cf03b4c-c779-49d0-be96-54963187bbb6\") " pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.772371 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.772294    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5"
Sep 05 07:49:52.884962 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.884929    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:52.977562 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.977520    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fb56df5ff-r8pwz"]
Sep 05 07:49:52.981095 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.981073    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:52.992484 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:52.992461    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fb56df5ff-r8pwz"]
Sep 05 07:49:53.032279 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.031752    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-serving-cert\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.032279 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.031933    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-oauth-config\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.032279 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.032000    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28mkz\" (UniqueName: \"kubernetes.io/projected/75d6343a-5041-4585-8fab-3cc85c560772-kube-api-access-28mkz\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.032279 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.032042    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-console-config\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.032279 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.032065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-trusted-ca-bundle\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.032279 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.032096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-oauth-serving-cert\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.032279 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.032157    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-service-ca\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.133053 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.133026    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28mkz\" (UniqueName: \"kubernetes.io/projected/75d6343a-5041-4585-8fab-3cc85c560772-kube-api-access-28mkz\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.133130 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.133083    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-console-config\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.133130 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.133109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-trusted-ca-bundle\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.133236 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.133137    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-oauth-serving-cert\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.133236 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.133184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-service-ca\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.133332 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.133247    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-serving-cert\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.133332 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.133292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-oauth-config\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.134747 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.134012    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-trusted-ca-bundle\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.134747 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.134144    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-console-config\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.134909 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.134772    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-oauth-serving-cert\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.135273 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.135232    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-service-ca\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.136395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.136372    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-oauth-config\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.137100 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.137082    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-serving-cert\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.151574 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.151497    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28mkz\" (UniqueName: \"kubernetes.io/projected/75d6343a-5041-4585-8fab-3cc85c560772-kube-api-access-28mkz\") pod \"console-fb56df5ff-r8pwz\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") " pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.262098 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.261800    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-769d67686c-7gbxm"]
Sep 05 07:49:53.298422 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.298396    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:49:53.332049 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.329756    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"]
Sep 05 07:49:53.347961 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.347873    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-596f58f6d6-k92n5"]
Sep 05 07:49:53.361918 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:53.361880    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf03b4c_c779_49d0_be96_54963187bbb6.slice/crio-f24fd93b2ebaa853df40b48e0f61ae24b4468d5ca24658282adc948e34ab863a WatchSource:0}: Error finding container f24fd93b2ebaa853df40b48e0f61ae24b4468d5ca24658282adc948e34ab863a: Status 404 returned error can't find the container with id f24fd93b2ebaa853df40b48e0f61ae24b4468d5ca24658282adc948e34ab863a
Sep 05 07:49:53.521490 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.521449    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 07:49:53.527887 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.527864    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.531167 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.530948    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 05 07:49:53.531167 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.530948    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 05 07:49:53.537545 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.531554    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 05 07:49:53.537545 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.531664    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 05 07:49:53.537545 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.531666    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 05 07:49:53.538254 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.538233    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 05 07:49:53.538653 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.538627    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 05 07:49:53.538736 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.538688    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 05 07:49:53.538873 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.538855    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 05 07:49:53.539024 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.539008    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fb62b\""
Sep 05 07:49:53.539299 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.539282    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 05 07:49:53.539544 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.539512    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 05 07:49:53.539633 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.538629    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 05 07:49:53.539852 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.539823    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-721anik807iek\""
Sep 05 07:49:53.543374 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.543186    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 05 07:49:53.546057 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.546038    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 05 07:49:53.547240 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.547129    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 07:49:53.634264 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.634235    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fb56df5ff-r8pwz"]
Sep 05 07:49:53.638891 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.638868    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaa167a5-c208-424e-acc7-9618be9cf172-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639005 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.638901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639005 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.638921    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaa167a5-c208-424e-acc7-9618be9cf172-config-out\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639005 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.638936    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639005 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.638959    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639208 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639208 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639034    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639208 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639081    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639208 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639107    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639208 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639208 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmzb9\" (UniqueName: \"kubernetes.io/projected/eaa167a5-c208-424e-acc7-9618be9cf172-kube-api-access-tmzb9\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639208 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-web-config\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639560 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639222    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639560 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639247    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639560 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639281    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639560 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639340    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639560 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639384    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639560 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639413    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-config\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639560 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639430    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.639560 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.639458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740480 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740439    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaa167a5-c208-424e-acc7-9618be9cf172-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740664 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740664 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaa167a5-c208-424e-acc7-9618be9cf172-config-out\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740664 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740572    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740664 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740598    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740664 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740664 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740664    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740697    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740735    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740782    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740812    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmzb9\" (UniqueName: \"kubernetes.io/projected/eaa167a5-c208-424e-acc7-9618be9cf172-kube-api-access-tmzb9\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740842    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-web-config\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740864    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.740978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.740970    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.741351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.741000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.741351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.741025    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.741351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.741055    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-config\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.741351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.741077    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.741351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.741118    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.741351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.741174    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.741351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.741208    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.741757 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.741731    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.743000 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.742965    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.745476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.744672    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.745476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.745150    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.745476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.745156    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.745795 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.745774    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.746768 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.746366    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eaa167a5-c208-424e-acc7-9618be9cf172-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.746768 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.746723    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eaa167a5-c208-424e-acc7-9618be9cf172-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.746931 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.746852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.748104 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.748064    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-web-config\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.750725 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.748837    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.750725 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.749111    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.750725 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.749519    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eaa167a5-c208-424e-acc7-9618be9cf172-config-out\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.750725 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.750343    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.750970 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.750793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.751292 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.751270    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.751408 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.751385    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/eaa167a5-c208-424e-acc7-9618be9cf172-config\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.754203 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.754180    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmzb9\" (UniqueName: \"kubernetes.io/projected/eaa167a5-c208-424e-acc7-9618be9cf172-kube-api-access-tmzb9\") pod \"prometheus-k8s-0\" (UID: \"eaa167a5-c208-424e-acc7-9618be9cf172\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.846438 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.846397    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:49:53.907286 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.907148    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw" event={"ID":"8329852e-bd3d-453d-b2ee-ae97d1b6f8cd","Type":"ContainerStarted","Data":"ef735aabac3c45010f45ec70476db147a9353444dee39c376e982467c94a0f0f"}
Sep 05 07:49:53.910813 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.910777    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh" event={"ID":"de5cc2c8-7bd9-4706-8788-8b8479a7087a","Type":"ContainerStarted","Data":"77469a632bc65674e9350eb421e7d76e018b75eed9fc2230a39ddf01c5a02af2"}
Sep 05 07:49:53.910942 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.910840    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh" event={"ID":"de5cc2c8-7bd9-4706-8788-8b8479a7087a","Type":"ContainerStarted","Data":"48667342f5df293b44077379f1e96d0966a2e9d9fbc1bebd70a9acdc48912103"}
Sep 05 07:49:53.910942 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.910872    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh" event={"ID":"de5cc2c8-7bd9-4706-8788-8b8479a7087a","Type":"ContainerStarted","Data":"55a768492d53fe1c7cd77b3fbc7b5d3c6bc28eaf4bab2fb2f7a5bbd62fc37e54"}
Sep 05 07:49:53.914036 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.913592    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w524v" event={"ID":"bab5ab31-4965-420e-a2e2-7c53dcd98c85","Type":"ContainerStarted","Data":"62b3ff8ac012314fe1bf34065b6440e5a7b8ed7f738c934723fbb05e896eea78"}
Sep 05 07:49:53.914036 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.913632    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w524v" event={"ID":"bab5ab31-4965-420e-a2e2-7c53dcd98c85","Type":"ContainerStarted","Data":"2b53cf0e0dd6a10b288143159e78b64feb7554b34a4183f04aebc93ccf940661"}
Sep 05 07:49:53.916366 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.916306    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv" event={"ID":"db7e756b-443f-469c-880a-8215b6ac72e3","Type":"ContainerStarted","Data":"bf70b0d5a7099e63be20b9acf855bfaf78309db64abe2c677ad5c6f7398ce0a4"}
Sep 05 07:49:53.918293 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.918221    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-769d67686c-7gbxm" event={"ID":"507dfce2-0001-4920-af9e-21af592e5a3c","Type":"ContainerStarted","Data":"656b602baa9aa070bda51f95f3028e2f105d3abb252f21c98cbf2dcf91e86cce"}
Sep 05 07:49:53.921291 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.921124    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph" event={"ID":"a7277c30-a1dd-4fbf-b05c-421072627753","Type":"ContainerStarted","Data":"9f597235c477c4ac3b7666601be07ccbcff15a6f00a0d183c109aa86f0c61968"}
Sep 05 07:49:53.921291 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.921194    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph" event={"ID":"a7277c30-a1dd-4fbf-b05c-421072627753","Type":"ContainerStarted","Data":"2abc89b7d4adac564f8a5cf3851670e614fe5ad5ab33f272c122ef58b7fe5815"}
Sep 05 07:49:53.921291 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.921237    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph" event={"ID":"a7277c30-a1dd-4fbf-b05c-421072627753","Type":"ContainerStarted","Data":"43a3dfcd19fa42ad8b8d16032a8d5d88f19e772223a9949c76c9003421a6a4e1"}
Sep 05 07:49:53.924449 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.923480    2569 generic.go:356] "Generic (PLEG): container finished" podID="20073340-5926-4f77-b1ff-f70558d61bb2" containerID="80ec9eb28566cb1baedfbf4bde3959b86cf3f14fc87a9c4c203e76e944db144d" exitCode=0
Sep 05 07:49:53.924449 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.923572    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20073340-5926-4f77-b1ff-f70558d61bb2","Type":"ContainerDied","Data":"80ec9eb28566cb1baedfbf4bde3959b86cf3f14fc87a9c4c203e76e944db144d"}
Sep 05 07:49:53.929153 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.928861    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-67csw" podStartSLOduration=4.596312025 podStartE2EDuration="6.928843481s" totalImagesPullingTime="2.332531456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:53.928194033 +0000 UTC m=+74.873515839" watchObservedRunningTime="2026-09-05 07:49:53.928843481 +0000 UTC m=+74.874165287"
Sep 05 07:49:53.932106 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.932007    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb56df5ff-r8pwz" event={"ID":"75d6343a-5041-4585-8fab-3cc85c560772","Type":"ContainerStarted","Data":"ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24"}
Sep 05 07:49:53.932106 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.932048    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb56df5ff-r8pwz" event={"ID":"75d6343a-5041-4585-8fab-3cc85c560772","Type":"ContainerStarted","Data":"5352b583cd99067c7389fd433a016ccfc920dd7c265997286f623cfbb3fba572"}
Sep 05 07:49:53.936706 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:53.936654    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5" event={"ID":"9cf03b4c-c779-49d0-be96-54963187bbb6","Type":"ContainerStarted","Data":"f24fd93b2ebaa853df40b48e0f61ae24b4468d5ca24658282adc948e34ab863a"}
Sep 05 07:49:54.006487 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:54.006426    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hhgqh" podStartSLOduration=4.467380868 podStartE2EDuration="7.006404446s" totalImagesPullingTime="2.539023578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:53.983730579 +0000 UTC m=+74.929052385" watchObservedRunningTime="2026-09-05 07:49:54.006404446 +0000 UTC m=+74.951726249"
Sep 05 07:49:54.008664 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:54.008619    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-w524v" podStartSLOduration=6.097114943 podStartE2EDuration="7.008604117s" totalImagesPullingTime="817.820432ms" totalInitContainerRuntime="93.668742ms" isStatefulPod=true podCreationTimestamp="2026-09-05 07:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:54.005555262 +0000 UTC m=+74.950877059" watchObservedRunningTime="2026-09-05 07:49:54.008604117 +0000 UTC m=+74.953925923"
Sep 05 07:49:54.029155 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:54.028879    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fb56df5ff-r8pwz" podStartSLOduration=2.028863948 podStartE2EDuration="2.028863948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:54.027758048 +0000 UTC m=+74.973079853" watchObservedRunningTime="2026-09-05 07:49:54.028863948 +0000 UTC m=+74.974185811"
Sep 05 07:49:54.164876 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:54.164845    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 07:49:54.942903 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:54.942863    2569 generic.go:356] "Generic (PLEG): container finished" podID="eaa167a5-c208-424e-acc7-9618be9cf172" containerID="67f46b7a6a888994530759849bbcfdd51c813357f7931ca56070b9e48803f90f" exitCode=0
Sep 05 07:49:54.943353 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:54.942961    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa167a5-c208-424e-acc7-9618be9cf172","Type":"ContainerDied","Data":"67f46b7a6a888994530759849bbcfdd51c813357f7931ca56070b9e48803f90f"}
Sep 05 07:49:54.943353 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:54.943080    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa167a5-c208-424e-acc7-9618be9cf172","Type":"ContainerStarted","Data":"12fa283a3681cb5af51636be3d0fd60a02c4be52e70158073dcda000ead4f317"}
Sep 05 07:49:54.946105 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:54.946079    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph" event={"ID":"a7277c30-a1dd-4fbf-b05c-421072627753","Type":"ContainerStarted","Data":"30c817ba3c7fb34d7fa417881d2cbf50692fa0608a6d325f011f145096bd3c75"}
Sep 05 07:49:56.965908 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.964984    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph" event={"ID":"a7277c30-a1dd-4fbf-b05c-421072627753","Type":"ContainerStarted","Data":"7a5e2fa92880099ed922f1deb8e8f4dc105523c01e80621abc81b729c4968197"}
Sep 05 07:49:56.965908 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.965068    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph" event={"ID":"a7277c30-a1dd-4fbf-b05c-421072627753","Type":"ContainerStarted","Data":"d7cb0f2b96a252e6e5cb6a35ee2b8d42da3ee7a01864020dc1e6faa5d95dc181"}
Sep 05 07:49:56.965908 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.965409    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:49:56.969936 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.969878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20073340-5926-4f77-b1ff-f70558d61bb2","Type":"ContainerStarted","Data":"059eae056a56ba2f1cd720de3d8024e5bed27536a82a985d46aa170ac811210e"}
Sep 05 07:49:56.969936 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.969928    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20073340-5926-4f77-b1ff-f70558d61bb2","Type":"ContainerStarted","Data":"caa06ad27ecd4a58405de68d7ea73190ffaf695dbcb778a73cba71c625ca5354"}
Sep 05 07:49:56.970155 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.969953    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20073340-5926-4f77-b1ff-f70558d61bb2","Type":"ContainerStarted","Data":"07f7a40e00a7fa0c1058fd0bc96edde1852cd0628dd00540678a1bb47b9b83af"}
Sep 05 07:49:56.970155 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.969992    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20073340-5926-4f77-b1ff-f70558d61bb2","Type":"ContainerStarted","Data":"4fe074b2d320e0d3190b6fbce47998179a6600f10fc54c68d3ec414d694c9e2d"}
Sep 05 07:49:56.973909 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.973871    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5" event={"ID":"9cf03b4c-c779-49d0-be96-54963187bbb6","Type":"ContainerStarted","Data":"b041737d98b58acbdecf1f7b53d00dd29d6f43b529ebad32a0a97961863873e1"}
Sep 05 07:49:56.974014 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.973932    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5" event={"ID":"9cf03b4c-c779-49d0-be96-54963187bbb6","Type":"ContainerStarted","Data":"a50f6f47c434704dc4e5cb5d6c9c9f26228389e9f41a304e0df3317f49b50d1f"}
Sep 05 07:49:56.974014 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.973965    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5" event={"ID":"9cf03b4c-c779-49d0-be96-54963187bbb6","Type":"ContainerStarted","Data":"1582c1db6943fc0612584b8b4c41881bc697ef4df4474c13f2d2de79659c1d13"}
Sep 05 07:49:56.977169 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.976841    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv" event={"ID":"db7e756b-443f-469c-880a-8215b6ac72e3","Type":"ContainerStarted","Data":"598e898cd7a0185af209d819d919366b095f94c7bf4462eaf7923ce4c6fd2ce6"}
Sep 05 07:49:56.977330 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.977307    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:56.979471 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.979451    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-769d67686c-7gbxm" event={"ID":"507dfce2-0001-4920-af9e-21af592e5a3c","Type":"ContainerStarted","Data":"236caf36fc17a94491ac73e659806692b662e06eb2f6276220d29fda612efc3c"}
Sep 05 07:49:56.984085 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.984069    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv"
Sep 05 07:49:56.994041 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:56.993994    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph" podStartSLOduration=3.635028347 podStartE2EDuration="6.993982154s" totalImagesPullingTime="3.358953807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:56.991653243 +0000 UTC m=+77.936975048" watchObservedRunningTime="2026-09-05 07:49:56.993982154 +0000 UTC m=+77.939303958"
Sep 05 07:49:57.012849 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.012802    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-769d67686c-7gbxm" podStartSLOduration=2.7825921129999998 podStartE2EDuration="6.012787574s" totalImagesPullingTime="3.230195461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:57.011638128 +0000 UTC m=+77.956959932" watchObservedRunningTime="2026-09-05 07:49:57.012787574 +0000 UTC m=+77.958109382"
Sep 05 07:49:57.031033 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.030990    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f2nwv" podStartSLOduration=2.870779029 podStartE2EDuration="6.03097569s" totalImagesPullingTime="3.160196661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:57.029972974 +0000 UTC m=+77.975294781" watchObservedRunningTime="2026-09-05 07:49:57.03097569 +0000 UTC m=+77.976297533"
Sep 05 07:49:57.061854 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.061800    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-596f58f6d6-k92n5" podStartSLOduration=1.926317893 podStartE2EDuration="5.061780081s" totalImagesPullingTime="3.135462188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:57.061334596 +0000 UTC m=+78.006656402" watchObservedRunningTime="2026-09-05 07:49:57.061780081 +0000 UTC m=+78.007101888"
Sep 05 07:49:57.161250 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.161214    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fb56df5ff-r8pwz"]
Sep 05 07:49:57.194636 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.194595    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c856d4b87-hbwb5"]
Sep 05 07:49:57.198400 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.198372    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.209453 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.209429    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c856d4b87-hbwb5"]
Sep 05 07:49:57.282378 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.282301    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-oauth-serving-cert\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.282378 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.282343    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-oauth-config\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.282616 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.282417    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-trusted-ca-bundle\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.282616 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.282464    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-serving-cert\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.282616 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.282486    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-console-config\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.282728 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.282609    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndbxp\" (UniqueName: \"kubernetes.io/projected/7a706539-f8b9-4cda-b426-55f16105cc2a-kube-api-access-ndbxp\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.282728 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.282663    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-service-ca\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.383958 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.383932    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndbxp\" (UniqueName: \"kubernetes.io/projected/7a706539-f8b9-4cda-b426-55f16105cc2a-kube-api-access-ndbxp\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.384125 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.383981    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-service-ca\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.384125 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.384018    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-oauth-serving-cert\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.384125 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.384046    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-oauth-config\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.384125 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.384096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-trusted-ca-bundle\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.384598 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.384573    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-serving-cert\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.384731 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.384610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-console-config\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.384815 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.384793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-service-ca\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.384978 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.384878    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-oauth-serving-cert\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.385150 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.385129    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-trusted-ca-bundle\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.385242 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.385222    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-console-config\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.387423 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.387404    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-oauth-config\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.387518 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.387432    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-serving-cert\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.393242 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.393225    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndbxp\" (UniqueName: \"kubernetes.io/projected/7a706539-f8b9-4cda-b426-55f16105cc2a-kube-api-access-ndbxp\") pod \"console-7c856d4b87-hbwb5\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") " pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.512232 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.512188    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:49:57.806345 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.805733    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c856d4b87-hbwb5"]
Sep 05 07:49:57.989993 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.989911    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20073340-5926-4f77-b1ff-f70558d61bb2","Type":"ContainerStarted","Data":"a9f9a83de60fbeb3955b8086383e820728dd204990f0c22ab896da07947f3c96"}
Sep 05 07:49:57.989993 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:57.989985    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20073340-5926-4f77-b1ff-f70558d61bb2","Type":"ContainerStarted","Data":"e07965d4c5b98cf9a7e2b3fffa284e2206df8ae5f46849f625afa84abe648b5f"}
Sep 05 07:49:58.019336 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:58.019282    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.019980767 podStartE2EDuration="10.019267175s" totalImagesPullingTime="4.970820528s" totalInitContainerRuntime="28.46588ms" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:49:58.017045716 +0000 UTC m=+78.962367535" watchObservedRunningTime="2026-09-05 07:49:58.019267175 +0000 UTC m=+78.964588979"
Sep 05 07:49:58.988419 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:49:58.988383    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a706539_f8b9_4cda_b426_55f16105cc2a.slice/crio-bb4a3d60141d3d47ca7ee3605f7f028369872df0947de271ae2f4861c1819c70 WatchSource:0}: Error finding container bb4a3d60141d3d47ca7ee3605f7f028369872df0947de271ae2f4861c1819c70: Status 404 returned error can't find the container with id bb4a3d60141d3d47ca7ee3605f7f028369872df0947de271ae2f4861c1819c70
Sep 05 07:49:58.995112 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:49:58.995089    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c856d4b87-hbwb5" event={"ID":"7a706539-f8b9-4cda-b426-55f16105cc2a","Type":"ContainerStarted","Data":"bb4a3d60141d3d47ca7ee3605f7f028369872df0947de271ae2f4861c1819c70"}
Sep 05 07:50:00.002977 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:00.002933    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa167a5-c208-424e-acc7-9618be9cf172","Type":"ContainerStarted","Data":"f701cf9356ec80feb56895647f669a8f1935d16833c3a53c74a16b2bcf84da75"}
Sep 05 07:50:00.003386 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:00.003019    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa167a5-c208-424e-acc7-9618be9cf172","Type":"ContainerStarted","Data":"48db25ad7be9eaa86702af1f7803490362171f8def1312997c42795c924de8e9"}
Sep 05 07:50:00.003386 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:00.003063    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa167a5-c208-424e-acc7-9618be9cf172","Type":"ContainerStarted","Data":"4f7829f949fccd75f0c3861f89c9fa6d274893c8cd06a1f9787d4fbd84cc77ab"}
Sep 05 07:50:00.003386 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:00.003101    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa167a5-c208-424e-acc7-9618be9cf172","Type":"ContainerStarted","Data":"146fd4188f5970872a37bca2047ef459697a0f1b91c56366a18e42e77f50e991"}
Sep 05 07:50:00.003386 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:00.003136    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa167a5-c208-424e-acc7-9618be9cf172","Type":"ContainerStarted","Data":"c0bc66f792239b226a08fbf5b41e00115b90c5deeacac134837326061c7eb533"}
Sep 05 07:50:00.003386 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:00.003171    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eaa167a5-c208-424e-acc7-9618be9cf172","Type":"ContainerStarted","Data":"417f56673261b8c386940213fd353fe22bdb8d57699ad654e9bfc252ab559bd6"}
Sep 05 07:50:00.004614 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:00.004591    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c856d4b87-hbwb5" event={"ID":"7a706539-f8b9-4cda-b426-55f16105cc2a","Type":"ContainerStarted","Data":"8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6"}
Sep 05 07:50:00.043895 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:00.043843    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.912051903 podStartE2EDuration="7.043827618s" totalImagesPullingTime="4.100293397s" totalInitContainerRuntime="31.482318ms" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:50:00.043582491 +0000 UTC m=+80.988904296" watchObservedRunningTime="2026-09-05 07:50:00.043827618 +0000 UTC m=+80.989149449"
Sep 05 07:50:00.063368 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:00.063318    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c856d4b87-hbwb5" podStartSLOduration=3.063302813 podStartE2EDuration="3.063302813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:49:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:50:00.062448088 +0000 UTC m=+81.007769893" watchObservedRunningTime="2026-09-05 07:50:00.063302813 +0000 UTC m=+81.008624617"
Sep 05 07:50:02.996616 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:02.996485    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8664b866fc-qlcph"
Sep 05 07:50:03.299559 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:03.299451    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:50:03.847563 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:03.847512    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:50:07.512472 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:07.512425    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:50:07.512893 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:07.512567    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:50:07.517390 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:07.517369    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:50:08.043097 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:08.043072    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:50:08.090256 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:08.090227    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-568f4bdd46-6lxfd"]
Sep 05 07:50:11.928131 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:11.928101    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:50:11.928503 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:11.928149    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:50:13.939055 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:13.938997    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74f48649c8-7b27q" podUID="f64b9381-137e-425f-961e-389e4d0b0f61" containerName="console" containerID="cri-o://76fb51544a01ce99f40bec355ceb1ac4a3e7fca15a047a6a49ac5388f776f579" gracePeriod=15
Sep 05 07:50:14.066969 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.066942    2569 generic.go:356] "Generic (PLEG): container finished" podID="f64b9381-137e-425f-961e-389e4d0b0f61" containerID="76fb51544a01ce99f40bec355ceb1ac4a3e7fca15a047a6a49ac5388f776f579" exitCode=0
Sep 05 07:50:14.067075 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.067014    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f48649c8-7b27q" event={"ID":"f64b9381-137e-425f-961e-389e4d0b0f61","Type":"ContainerDied","Data":"76fb51544a01ce99f40bec355ceb1ac4a3e7fca15a047a6a49ac5388f776f579"}
Sep 05 07:50:14.190887 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.190835    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:50:14.343463 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.343433    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-console-config\") pod \"f64b9381-137e-425f-961e-389e4d0b0f61\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") "
Sep 05 07:50:14.343627 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.343474    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbdj4\" (UniqueName: \"kubernetes.io/projected/f64b9381-137e-425f-961e-389e4d0b0f61-kube-api-access-wbdj4\") pod \"f64b9381-137e-425f-961e-389e4d0b0f61\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") "
Sep 05 07:50:14.343627 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.343507    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-oauth-serving-cert\") pod \"f64b9381-137e-425f-961e-389e4d0b0f61\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") "
Sep 05 07:50:14.343627 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.343547    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-serving-cert\") pod \"f64b9381-137e-425f-961e-389e4d0b0f61\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") "
Sep 05 07:50:14.343627 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.343574    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-oauth-config\") pod \"f64b9381-137e-425f-961e-389e4d0b0f61\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") "
Sep 05 07:50:14.343627 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.343591    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-service-ca\") pod \"f64b9381-137e-425f-961e-389e4d0b0f61\" (UID: \"f64b9381-137e-425f-961e-389e4d0b0f61\") "
Sep 05 07:50:14.344192 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.343918    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-console-config" (OuterVolumeSpecName: "console-config") pod "f64b9381-137e-425f-961e-389e4d0b0f61" (UID: "f64b9381-137e-425f-961e-389e4d0b0f61"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:14.344192 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.343901    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f64b9381-137e-425f-961e-389e4d0b0f61" (UID: "f64b9381-137e-425f-961e-389e4d0b0f61"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:14.344192 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.344066    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-service-ca" (OuterVolumeSpecName: "service-ca") pod "f64b9381-137e-425f-961e-389e4d0b0f61" (UID: "f64b9381-137e-425f-961e-389e4d0b0f61"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:14.346050 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.346021    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f64b9381-137e-425f-961e-389e4d0b0f61-kube-api-access-wbdj4" (OuterVolumeSpecName: "kube-api-access-wbdj4") pod "f64b9381-137e-425f-961e-389e4d0b0f61" (UID: "f64b9381-137e-425f-961e-389e4d0b0f61"). InnerVolumeSpecName "kube-api-access-wbdj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:50:14.346150 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.346037    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f64b9381-137e-425f-961e-389e4d0b0f61" (UID: "f64b9381-137e-425f-961e-389e4d0b0f61"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:50:14.346150 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.346080    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f64b9381-137e-425f-961e-389e4d0b0f61" (UID: "f64b9381-137e-425f-961e-389e4d0b0f61"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:50:14.444715 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.444663    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-console-config\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:14.444715 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.444685    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbdj4\" (UniqueName: \"kubernetes.io/projected/f64b9381-137e-425f-961e-389e4d0b0f61-kube-api-access-wbdj4\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:14.444715 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.444695    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-oauth-serving-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:14.444715 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.444703    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-serving-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:14.444715 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.444711    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f64b9381-137e-425f-961e-389e4d0b0f61-console-oauth-config\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:14.444913 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:14.444719    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f64b9381-137e-425f-961e-389e4d0b0f61-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:15.072784 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:15.072757    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f48649c8-7b27q"
Sep 05 07:50:15.073168 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:15.072750    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f48649c8-7b27q" event={"ID":"f64b9381-137e-425f-961e-389e4d0b0f61","Type":"ContainerDied","Data":"d89ccc8b0879d1d9c99e19c531935fff188c466ffd49959d14172d20f6c0f5d2"}
Sep 05 07:50:15.073168 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:15.072897    2569 scope.go:117] "RemoveContainer" containerID="76fb51544a01ce99f40bec355ceb1ac4a3e7fca15a047a6a49ac5388f776f579"
Sep 05 07:50:15.093755 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:15.093729    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74f48649c8-7b27q"]
Sep 05 07:50:15.099820 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:15.099800    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74f48649c8-7b27q"]
Sep 05 07:50:15.569387 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:15.569361    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f64b9381-137e-425f-961e-389e4d0b0f61" path="/var/lib/kubelet/pods/f64b9381-137e-425f-961e-389e4d0b0f61/volumes"
Sep 05 07:50:21.890901 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:21.890873    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7njm4"
Sep 05 07:50:22.187657 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.187518    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fb56df5ff-r8pwz" podUID="75d6343a-5041-4585-8fab-3cc85c560772" containerName="console" containerID="cri-o://ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24" gracePeriod=15
Sep 05 07:50:22.475049 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.475025    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:50:22.519484 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.519459    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-oauth-config\") pod \"75d6343a-5041-4585-8fab-3cc85c560772\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") "
Sep 05 07:50:22.519484 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.519488    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-trusted-ca-bundle\") pod \"75d6343a-5041-4585-8fab-3cc85c560772\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") "
Sep 05 07:50:22.519717 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.519510    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-service-ca\") pod \"75d6343a-5041-4585-8fab-3cc85c560772\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") "
Sep 05 07:50:22.519717 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.519583    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28mkz\" (UniqueName: \"kubernetes.io/projected/75d6343a-5041-4585-8fab-3cc85c560772-kube-api-access-28mkz\") pod \"75d6343a-5041-4585-8fab-3cc85c560772\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") "
Sep 05 07:50:22.519717 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.519609    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-console-config\") pod \"75d6343a-5041-4585-8fab-3cc85c560772\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") "
Sep 05 07:50:22.519717 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.519648    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-serving-cert\") pod \"75d6343a-5041-4585-8fab-3cc85c560772\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") "
Sep 05 07:50:22.519717 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.519675    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-oauth-serving-cert\") pod \"75d6343a-5041-4585-8fab-3cc85c560772\" (UID: \"75d6343a-5041-4585-8fab-3cc85c560772\") "
Sep 05 07:50:22.520167 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.520033    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "75d6343a-5041-4585-8fab-3cc85c560772" (UID: "75d6343a-5041-4585-8fab-3cc85c560772"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:22.520167 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.520132    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-console-config" (OuterVolumeSpecName: "console-config") pod "75d6343a-5041-4585-8fab-3cc85c560772" (UID: "75d6343a-5041-4585-8fab-3cc85c560772"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:22.520167 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.520114    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-service-ca" (OuterVolumeSpecName: "service-ca") pod "75d6343a-5041-4585-8fab-3cc85c560772" (UID: "75d6343a-5041-4585-8fab-3cc85c560772"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:22.520382 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.520258    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "75d6343a-5041-4585-8fab-3cc85c560772" (UID: "75d6343a-5041-4585-8fab-3cc85c560772"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:22.521837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.521804    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "75d6343a-5041-4585-8fab-3cc85c560772" (UID: "75d6343a-5041-4585-8fab-3cc85c560772"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:50:22.521837 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.521824    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "75d6343a-5041-4585-8fab-3cc85c560772" (UID: "75d6343a-5041-4585-8fab-3cc85c560772"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:50:22.521953 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.521867    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75d6343a-5041-4585-8fab-3cc85c560772-kube-api-access-28mkz" (OuterVolumeSpecName: "kube-api-access-28mkz") pod "75d6343a-5041-4585-8fab-3cc85c560772" (UID: "75d6343a-5041-4585-8fab-3cc85c560772"). InnerVolumeSpecName "kube-api-access-28mkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:50:22.620804 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.620774    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-oauth-config\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:22.620804 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.620799    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-trusted-ca-bundle\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:22.620804 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.620810    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:22.620995 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.620818    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28mkz\" (UniqueName: \"kubernetes.io/projected/75d6343a-5041-4585-8fab-3cc85c560772-kube-api-access-28mkz\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:22.620995 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.620827    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-console-config\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:22.620995 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.620835    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75d6343a-5041-4585-8fab-3cc85c560772-console-serving-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:22.620995 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:22.620842    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75d6343a-5041-4585-8fab-3cc85c560772-oauth-serving-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:23.114466 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:23.114430    2569 generic.go:356] "Generic (PLEG): container finished" podID="75d6343a-5041-4585-8fab-3cc85c560772" containerID="ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24" exitCode=0
Sep 05 07:50:23.114876 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:23.114481    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fb56df5ff-r8pwz"
Sep 05 07:50:23.114876 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:23.114538    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb56df5ff-r8pwz" event={"ID":"75d6343a-5041-4585-8fab-3cc85c560772","Type":"ContainerDied","Data":"ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24"}
Sep 05 07:50:23.114876 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:23.114593    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fb56df5ff-r8pwz" event={"ID":"75d6343a-5041-4585-8fab-3cc85c560772","Type":"ContainerDied","Data":"5352b583cd99067c7389fd433a016ccfc920dd7c265997286f623cfbb3fba572"}
Sep 05 07:50:23.114876 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:23.114614    2569 scope.go:117] "RemoveContainer" containerID="ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24"
Sep 05 07:50:23.123991 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:23.123973    2569 scope.go:117] "RemoveContainer" containerID="ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24"
Sep 05 07:50:23.124265 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:50:23.124247    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24\": container with ID starting with ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24 not found: ID does not exist" containerID="ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24"
Sep 05 07:50:23.124333 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:23.124277    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24"} err="failed to get container status \"ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24\": rpc error: code = NotFound desc = could not find container \"ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24\": container with ID starting with ed19599cb21e37512bd0cda7ed9f97ca56d4abcdcf19fcf41717bd215fb25e24 not found: ID does not exist"
Sep 05 07:50:23.134347 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:23.134327    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fb56df5ff-r8pwz"]
Sep 05 07:50:23.138358 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:23.138340    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fb56df5ff-r8pwz"]
Sep 05 07:50:23.569908 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:23.569878    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75d6343a-5041-4585-8fab-3cc85c560772" path="/var/lib/kubelet/pods/75d6343a-5041-4585-8fab-3cc85c560772/volumes"
Sep 05 07:50:31.933588 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:31.933553    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:50:31.938047 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:31.938025    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-769d67686c-7gbxm"
Sep 05 07:50:33.112471 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.112413    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-568f4bdd46-6lxfd" podUID="49b3d61c-e946-48c6-8584-9b8680a3149f" containerName="console" containerID="cri-o://f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda" gracePeriod=15
Sep 05 07:50:33.357787 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.357763    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:50:33.419301 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.419222    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-console-config\") pod \"49b3d61c-e946-48c6-8584-9b8680a3149f\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") "
Sep 05 07:50:33.419301 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.419261    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-oauth-serving-cert\") pod \"49b3d61c-e946-48c6-8584-9b8680a3149f\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") "
Sep 05 07:50:33.419301 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.419281    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-oauth-config\") pod \"49b3d61c-e946-48c6-8584-9b8680a3149f\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") "
Sep 05 07:50:33.419301 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.419299    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2lfb\" (UniqueName: \"kubernetes.io/projected/49b3d61c-e946-48c6-8584-9b8680a3149f-kube-api-access-s2lfb\") pod \"49b3d61c-e946-48c6-8584-9b8680a3149f\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") "
Sep 05 07:50:33.419625 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.419374    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-service-ca\") pod \"49b3d61c-e946-48c6-8584-9b8680a3149f\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") "
Sep 05 07:50:33.419625 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.419407    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-serving-cert\") pod \"49b3d61c-e946-48c6-8584-9b8680a3149f\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") "
Sep 05 07:50:33.419625 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.419461    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-trusted-ca-bundle\") pod \"49b3d61c-e946-48c6-8584-9b8680a3149f\" (UID: \"49b3d61c-e946-48c6-8584-9b8680a3149f\") "
Sep 05 07:50:33.419779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.419631    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-console-config" (OuterVolumeSpecName: "console-config") pod "49b3d61c-e946-48c6-8584-9b8680a3149f" (UID: "49b3d61c-e946-48c6-8584-9b8680a3149f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:33.419779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.419666    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "49b3d61c-e946-48c6-8584-9b8680a3149f" (UID: "49b3d61c-e946-48c6-8584-9b8680a3149f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:33.419779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.419707    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-console-config\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:33.420133 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.420110    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-service-ca" (OuterVolumeSpecName: "service-ca") pod "49b3d61c-e946-48c6-8584-9b8680a3149f" (UID: "49b3d61c-e946-48c6-8584-9b8680a3149f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:33.420217 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.420186    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "49b3d61c-e946-48c6-8584-9b8680a3149f" (UID: "49b3d61c-e946-48c6-8584-9b8680a3149f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:50:33.421770 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.421739    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "49b3d61c-e946-48c6-8584-9b8680a3149f" (UID: "49b3d61c-e946-48c6-8584-9b8680a3149f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:50:33.421871 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.421771    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49b3d61c-e946-48c6-8584-9b8680a3149f-kube-api-access-s2lfb" (OuterVolumeSpecName: "kube-api-access-s2lfb") pod "49b3d61c-e946-48c6-8584-9b8680a3149f" (UID: "49b3d61c-e946-48c6-8584-9b8680a3149f"). InnerVolumeSpecName "kube-api-access-s2lfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:50:33.421871 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.421787    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "49b3d61c-e946-48c6-8584-9b8680a3149f" (UID: "49b3d61c-e946-48c6-8584-9b8680a3149f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:50:33.520977 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.520948    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:33.520977 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.520974    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-serving-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:33.520977 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.520983    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-trusted-ca-bundle\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:33.521182 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.520992    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49b3d61c-e946-48c6-8584-9b8680a3149f-oauth-serving-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:33.521182 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.521000    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49b3d61c-e946-48c6-8584-9b8680a3149f-console-oauth-config\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:33.521182 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:33.521008    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2lfb\" (UniqueName: \"kubernetes.io/projected/49b3d61c-e946-48c6-8584-9b8680a3149f-kube-api-access-s2lfb\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:50:34.168717 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:34.168680    2569 generic.go:356] "Generic (PLEG): container finished" podID="49b3d61c-e946-48c6-8584-9b8680a3149f" containerID="f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda" exitCode=0
Sep 05 07:50:34.169197 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:34.168750    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568f4bdd46-6lxfd"
Sep 05 07:50:34.169197 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:34.168766    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568f4bdd46-6lxfd" event={"ID":"49b3d61c-e946-48c6-8584-9b8680a3149f","Type":"ContainerDied","Data":"f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda"}
Sep 05 07:50:34.169197 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:34.168847    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568f4bdd46-6lxfd" event={"ID":"49b3d61c-e946-48c6-8584-9b8680a3149f","Type":"ContainerDied","Data":"a5a4ce29550592804eeaadaddaeee1ecf514ec00e55df31f2b675bda5121472f"}
Sep 05 07:50:34.169197 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:34.168870    2569 scope.go:117] "RemoveContainer" containerID="f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda"
Sep 05 07:50:34.177481 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:34.177461    2569 scope.go:117] "RemoveContainer" containerID="f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda"
Sep 05 07:50:34.177788 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:50:34.177767    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda\": container with ID starting with f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda not found: ID does not exist" containerID="f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda"
Sep 05 07:50:34.177861 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:34.177795    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda"} err="failed to get container status \"f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda\": rpc error: code = NotFound desc = could not find container \"f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda\": container with ID starting with f70bb27f287ff5ba3e98e856a07013bb7c93ba2acb3a32f6917b111955a73bda not found: ID does not exist"
Sep 05 07:50:34.187269 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:34.187249    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-568f4bdd46-6lxfd"]
Sep 05 07:50:34.191328 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:34.191310    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-568f4bdd46-6lxfd"]
Sep 05 07:50:35.569536 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:35.569495    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49b3d61c-e946-48c6-8584-9b8680a3149f" path="/var/lib/kubelet/pods/49b3d61c-e946-48c6-8584-9b8680a3149f/volumes"
Sep 05 07:50:53.847300 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:53.847267    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:50:53.866283 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:53.866262    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:50:54.265927 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:50:54.265853    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 07:51:22.368332 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.368301    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c486d8d79-ct8wh"]
Sep 05 07:51:22.368707 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.368628    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75d6343a-5041-4585-8fab-3cc85c560772" containerName="console"
Sep 05 07:51:22.368707 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.368640    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="75d6343a-5041-4585-8fab-3cc85c560772" containerName="console"
Sep 05 07:51:22.368707 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.368652    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64b9381-137e-425f-961e-389e4d0b0f61" containerName="console"
Sep 05 07:51:22.368707 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.368657    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64b9381-137e-425f-961e-389e4d0b0f61" containerName="console"
Sep 05 07:51:22.368707 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.368679    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49b3d61c-e946-48c6-8584-9b8680a3149f" containerName="console"
Sep 05 07:51:22.368707 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.368684    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49b3d61c-e946-48c6-8584-9b8680a3149f" containerName="console"
Sep 05 07:51:22.368887 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.368728    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="75d6343a-5041-4585-8fab-3cc85c560772" containerName="console"
Sep 05 07:51:22.368887 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.368737    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f64b9381-137e-425f-961e-389e4d0b0f61" containerName="console"
Sep 05 07:51:22.368887 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.368746    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="49b3d61c-e946-48c6-8584-9b8680a3149f" containerName="console"
Sep 05 07:51:22.372011 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.371993    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.380756 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.380729    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c486d8d79-ct8wh"]
Sep 05 07:51:22.437764 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.437739    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-oauth-serving-cert\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.437862 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.437791    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkpgz\" (UniqueName: \"kubernetes.io/projected/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-kube-api-access-dkpgz\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.437862 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.437812    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-oauth-config\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.437862 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.437841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-service-ca\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.437964 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.437868    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-serving-cert\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.437964 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.437932    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-config\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.437964 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.437956    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-trusted-ca-bundle\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.539125 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.539098    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkpgz\" (UniqueName: \"kubernetes.io/projected/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-kube-api-access-dkpgz\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.539231 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.539130    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-oauth-config\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.539231 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.539146    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-service-ca\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.539231 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.539165    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-serving-cert\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.539231 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.539192    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-config\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.539427 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.539312    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-trusted-ca-bundle\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.539427 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.539408    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-oauth-serving-cert\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.539963 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.539931    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-service-ca\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.540090 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.540069    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-config\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.540137 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.540069    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-oauth-serving-cert\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.540465 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.540441    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-trusted-ca-bundle\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.541767 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.541742    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-serving-cert\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.541860 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.541806    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-oauth-config\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.547944 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.547928    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkpgz\" (UniqueName: \"kubernetes.io/projected/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-kube-api-access-dkpgz\") pod \"console-5c486d8d79-ct8wh\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") " pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.680981 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.680931    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:22.959911 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:22.959894    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c486d8d79-ct8wh"]
Sep 05 07:51:22.962355 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:51:22.962327    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d4ede85_6c3e_4b47_91ef_a0e6cf90c2d4.slice/crio-70f3786493e858ea58ab35aaa67bcafaf0831f284f9bc60c39cd0391ed445e67 WatchSource:0}: Error finding container 70f3786493e858ea58ab35aaa67bcafaf0831f284f9bc60c39cd0391ed445e67: Status 404 returned error can't find the container with id 70f3786493e858ea58ab35aaa67bcafaf0831f284f9bc60c39cd0391ed445e67
Sep 05 07:51:23.364472 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:23.364438    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c486d8d79-ct8wh" event={"ID":"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4","Type":"ContainerStarted","Data":"ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514"}
Sep 05 07:51:23.364641 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:23.364486    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c486d8d79-ct8wh" event={"ID":"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4","Type":"ContainerStarted","Data":"70f3786493e858ea58ab35aaa67bcafaf0831f284f9bc60c39cd0391ed445e67"}
Sep 05 07:51:23.383545 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:23.383332    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c486d8d79-ct8wh" podStartSLOduration=1.3833141740000001 podStartE2EDuration="1.383314174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:51:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:51:23.383299893 +0000 UTC m=+164.328621698" watchObservedRunningTime="2026-09-05 07:51:23.383314174 +0000 UTC m=+164.328635975"
Sep 05 07:51:32.682089 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:32.682000    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:32.682089 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:32.682055    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:32.686964 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:32.686940    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:33.409658 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:33.409628    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:51:33.448822 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:33.448784    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c856d4b87-hbwb5"]
Sep 05 07:51:58.469577 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.469498    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c856d4b87-hbwb5" podUID="7a706539-f8b9-4cda-b426-55f16105cc2a" containerName="console" containerID="cri-o://8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6" gracePeriod=15
Sep 05 07:51:58.709013 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.708991    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:51:58.838182 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.838150    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-oauth-config\") pod \"7a706539-f8b9-4cda-b426-55f16105cc2a\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") "
Sep 05 07:51:58.838360 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.838205    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndbxp\" (UniqueName: \"kubernetes.io/projected/7a706539-f8b9-4cda-b426-55f16105cc2a-kube-api-access-ndbxp\") pod \"7a706539-f8b9-4cda-b426-55f16105cc2a\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") "
Sep 05 07:51:58.838360 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.838238    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-oauth-serving-cert\") pod \"7a706539-f8b9-4cda-b426-55f16105cc2a\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") "
Sep 05 07:51:58.838474 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.838435    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-serving-cert\") pod \"7a706539-f8b9-4cda-b426-55f16105cc2a\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") "
Sep 05 07:51:58.838553 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.838476    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-console-config\") pod \"7a706539-f8b9-4cda-b426-55f16105cc2a\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") "
Sep 05 07:51:58.838623 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.838554    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-service-ca\") pod \"7a706539-f8b9-4cda-b426-55f16105cc2a\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") "
Sep 05 07:51:58.838623 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.838586    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-trusted-ca-bundle\") pod \"7a706539-f8b9-4cda-b426-55f16105cc2a\" (UID: \"7a706539-f8b9-4cda-b426-55f16105cc2a\") "
Sep 05 07:51:58.838721 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.838618    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7a706539-f8b9-4cda-b426-55f16105cc2a" (UID: "7a706539-f8b9-4cda-b426-55f16105cc2a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:51:58.838899 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.838868    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-console-config" (OuterVolumeSpecName: "console-config") pod "7a706539-f8b9-4cda-b426-55f16105cc2a" (UID: "7a706539-f8b9-4cda-b426-55f16105cc2a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:51:58.838899 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.838894    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-oauth-serving-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:51:58.839214 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.839186    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-service-ca" (OuterVolumeSpecName: "service-ca") pod "7a706539-f8b9-4cda-b426-55f16105cc2a" (UID: "7a706539-f8b9-4cda-b426-55f16105cc2a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:51:58.839315 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.839293    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7a706539-f8b9-4cda-b426-55f16105cc2a" (UID: "7a706539-f8b9-4cda-b426-55f16105cc2a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:51:58.840604 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.840577    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7a706539-f8b9-4cda-b426-55f16105cc2a" (UID: "7a706539-f8b9-4cda-b426-55f16105cc2a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:51:58.840678 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.840608    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7a706539-f8b9-4cda-b426-55f16105cc2a" (UID: "7a706539-f8b9-4cda-b426-55f16105cc2a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:51:58.840678 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.840648    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a706539-f8b9-4cda-b426-55f16105cc2a-kube-api-access-ndbxp" (OuterVolumeSpecName: "kube-api-access-ndbxp") pod "7a706539-f8b9-4cda-b426-55f16105cc2a" (UID: "7a706539-f8b9-4cda-b426-55f16105cc2a"). InnerVolumeSpecName "kube-api-access-ndbxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:51:58.939988 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.939955    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndbxp\" (UniqueName: \"kubernetes.io/projected/7a706539-f8b9-4cda-b426-55f16105cc2a-kube-api-access-ndbxp\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:51:58.939988 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.939981    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-serving-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:51:58.939988 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.939991    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-console-config\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:51:58.939988 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.939999    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:51:58.940237 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.940007    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a706539-f8b9-4cda-b426-55f16105cc2a-trusted-ca-bundle\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:51:58.940237 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:58.940014    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a706539-f8b9-4cda-b426-55f16105cc2a-console-oauth-config\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:51:59.509612 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:59.509574    2569 generic.go:356] "Generic (PLEG): container finished" podID="7a706539-f8b9-4cda-b426-55f16105cc2a" containerID="8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6" exitCode=0
Sep 05 07:51:59.510035 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:59.509624    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c856d4b87-hbwb5"
Sep 05 07:51:59.510035 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:59.509623    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c856d4b87-hbwb5" event={"ID":"7a706539-f8b9-4cda-b426-55f16105cc2a","Type":"ContainerDied","Data":"8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6"}
Sep 05 07:51:59.510035 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:59.509760    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c856d4b87-hbwb5" event={"ID":"7a706539-f8b9-4cda-b426-55f16105cc2a","Type":"ContainerDied","Data":"bb4a3d60141d3d47ca7ee3605f7f028369872df0947de271ae2f4861c1819c70"}
Sep 05 07:51:59.510035 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:59.509786    2569 scope.go:117] "RemoveContainer" containerID="8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6"
Sep 05 07:51:59.518665 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:59.518564    2569 scope.go:117] "RemoveContainer" containerID="8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6"
Sep 05 07:51:59.518858 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:51:59.518840    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6\": container with ID starting with 8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6 not found: ID does not exist" containerID="8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6"
Sep 05 07:51:59.518903 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:59.518868    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6"} err="failed to get container status \"8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6\": rpc error: code = NotFound desc = could not find container \"8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6\": container with ID starting with 8488a235be25bdc34088f7f3a82513405459aa02d402a83dcbe31857b4712fa6 not found: ID does not exist"
Sep 05 07:51:59.531374 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:59.531352    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c856d4b87-hbwb5"]
Sep 05 07:51:59.536712 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:59.536692    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c856d4b87-hbwb5"]
Sep 05 07:51:59.568840 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:51:59.568820    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a706539-f8b9-4cda-b426-55f16105cc2a" path="/var/lib/kubelet/pods/7a706539-f8b9-4cda-b426-55f16105cc2a/volumes"
Sep 05 07:52:43.733602 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.733561    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"]
Sep 05 07:52:43.734013 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.733932    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a706539-f8b9-4cda-b426-55f16105cc2a" containerName="console"
Sep 05 07:52:43.734013 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.733948    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a706539-f8b9-4cda-b426-55f16105cc2a" containerName="console"
Sep 05 07:52:43.734013 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.734010    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a706539-f8b9-4cda-b426-55f16105cc2a" containerName="console"
Sep 05 07:52:43.738134 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.738119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"
Sep 05 07:52:43.740878 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.740846    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 05 07:52:43.740998 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.740884    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lkszg\""
Sep 05 07:52:43.740998 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.740930    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 05 07:52:43.748060 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.748034    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"]
Sep 05 07:52:43.891854 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.891818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a60e94c4-52e1-4b2d-93a5-a4573079c3dd-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cghrp\" (UID: \"a60e94c4-52e1-4b2d-93a5-a4573079c3dd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"
Sep 05 07:52:43.891854 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.891852    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s5m9\" (UniqueName: \"kubernetes.io/projected/a60e94c4-52e1-4b2d-93a5-a4573079c3dd-kube-api-access-7s5m9\") pod \"cert-manager-cainjector-559cdd8c87-cghrp\" (UID: \"a60e94c4-52e1-4b2d-93a5-a4573079c3dd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"
Sep 05 07:52:43.993214 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.993117    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a60e94c4-52e1-4b2d-93a5-a4573079c3dd-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cghrp\" (UID: \"a60e94c4-52e1-4b2d-93a5-a4573079c3dd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"
Sep 05 07:52:43.993214 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:43.993157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s5m9\" (UniqueName: \"kubernetes.io/projected/a60e94c4-52e1-4b2d-93a5-a4573079c3dd-kube-api-access-7s5m9\") pod \"cert-manager-cainjector-559cdd8c87-cghrp\" (UID: \"a60e94c4-52e1-4b2d-93a5-a4573079c3dd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"
Sep 05 07:52:44.002085 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:44.002054    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a60e94c4-52e1-4b2d-93a5-a4573079c3dd-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cghrp\" (UID: \"a60e94c4-52e1-4b2d-93a5-a4573079c3dd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"
Sep 05 07:52:44.002278 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:44.002248    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s5m9\" (UniqueName: \"kubernetes.io/projected/a60e94c4-52e1-4b2d-93a5-a4573079c3dd-kube-api-access-7s5m9\") pod \"cert-manager-cainjector-559cdd8c87-cghrp\" (UID: \"a60e94c4-52e1-4b2d-93a5-a4573079c3dd\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"
Sep 05 07:52:44.054135 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:44.054100    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"
Sep 05 07:52:44.321165 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:44.321119    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cghrp"]
Sep 05 07:52:44.323844 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:52:44.323817    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda60e94c4_52e1_4b2d_93a5_a4573079c3dd.slice/crio-814ac3b610fc01b09b6353509aa3e74816f8d76d658d90142ca67c6477d89350 WatchSource:0}: Error finding container 814ac3b610fc01b09b6353509aa3e74816f8d76d658d90142ca67c6477d89350: Status 404 returned error can't find the container with id 814ac3b610fc01b09b6353509aa3e74816f8d76d658d90142ca67c6477d89350
Sep 05 07:52:44.699495 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:44.699417    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp" event={"ID":"a60e94c4-52e1-4b2d-93a5-a4573079c3dd","Type":"ContainerStarted","Data":"814ac3b610fc01b09b6353509aa3e74816f8d76d658d90142ca67c6477d89350"}
Sep 05 07:52:47.724682 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:47.724634    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp" event={"ID":"a60e94c4-52e1-4b2d-93a5-a4573079c3dd","Type":"ContainerStarted","Data":"991f0eacb34e7244dac92e0f5c2c49b56128dafd6370d1c32a981bf5719874ad"}
Sep 05 07:52:47.743259 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:47.743202    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cghrp" podStartSLOduration=1.822722715 podStartE2EDuration="4.743186934s" totalImagesPullingTime="2.920464219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:52:47.74173149 +0000 UTC m=+248.687053296" watchObservedRunningTime="2026-09-05 07:52:47.743186934 +0000 UTC m=+248.688508738"
Sep 05 07:52:50.566361 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.566316    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-chbv7"]
Sep 05 07:52:50.569723 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.569706    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-chbv7"
Sep 05 07:52:50.572261 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.572242    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-6t69q\""
Sep 05 07:52:50.579295 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.579269    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-chbv7"]
Sep 05 07:52:50.650236 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.650197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x7sq\" (UniqueName: \"kubernetes.io/projected/412be205-59d8-48dc-b16e-7bc938c08a3b-kube-api-access-4x7sq\") pod \"cert-manager-55b4d954d5-chbv7\" (UID: \"412be205-59d8-48dc-b16e-7bc938c08a3b\") " pod="cert-manager/cert-manager-55b4d954d5-chbv7"
Sep 05 07:52:50.650398 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.650284    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/412be205-59d8-48dc-b16e-7bc938c08a3b-bound-sa-token\") pod \"cert-manager-55b4d954d5-chbv7\" (UID: \"412be205-59d8-48dc-b16e-7bc938c08a3b\") " pod="cert-manager/cert-manager-55b4d954d5-chbv7"
Sep 05 07:52:50.750910 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.750872    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x7sq\" (UniqueName: \"kubernetes.io/projected/412be205-59d8-48dc-b16e-7bc938c08a3b-kube-api-access-4x7sq\") pod \"cert-manager-55b4d954d5-chbv7\" (UID: \"412be205-59d8-48dc-b16e-7bc938c08a3b\") " pod="cert-manager/cert-manager-55b4d954d5-chbv7"
Sep 05 07:52:50.751090 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.750980    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/412be205-59d8-48dc-b16e-7bc938c08a3b-bound-sa-token\") pod \"cert-manager-55b4d954d5-chbv7\" (UID: \"412be205-59d8-48dc-b16e-7bc938c08a3b\") " pod="cert-manager/cert-manager-55b4d954d5-chbv7"
Sep 05 07:52:50.760230 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.760198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x7sq\" (UniqueName: \"kubernetes.io/projected/412be205-59d8-48dc-b16e-7bc938c08a3b-kube-api-access-4x7sq\") pod \"cert-manager-55b4d954d5-chbv7\" (UID: \"412be205-59d8-48dc-b16e-7bc938c08a3b\") " pod="cert-manager/cert-manager-55b4d954d5-chbv7"
Sep 05 07:52:50.760230 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.760217    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/412be205-59d8-48dc-b16e-7bc938c08a3b-bound-sa-token\") pod \"cert-manager-55b4d954d5-chbv7\" (UID: \"412be205-59d8-48dc-b16e-7bc938c08a3b\") " pod="cert-manager/cert-manager-55b4d954d5-chbv7"
Sep 05 07:52:50.879340 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:50.879299    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-chbv7"
Sep 05 07:52:51.147355 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:51.147330    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-chbv7"]
Sep 05 07:52:51.149931 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:52:51.149911    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod412be205_59d8_48dc_b16e_7bc938c08a3b.slice/crio-ee93cc3889f2c41945ff930b4a80219f8b18be6445a32c5fa82c49ae608e9044 WatchSource:0}: Error finding container ee93cc3889f2c41945ff930b4a80219f8b18be6445a32c5fa82c49ae608e9044: Status 404 returned error can't find the container with id ee93cc3889f2c41945ff930b4a80219f8b18be6445a32c5fa82c49ae608e9044
Sep 05 07:52:51.742583 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:51.742538    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-chbv7" event={"ID":"412be205-59d8-48dc-b16e-7bc938c08a3b","Type":"ContainerStarted","Data":"688cbefebb55b9aab457096323d75ac0c5a0b4f15ae4bd53c11b9f12978b60fb"}
Sep 05 07:52:51.742583 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:51.742579    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-chbv7" event={"ID":"412be205-59d8-48dc-b16e-7bc938c08a3b","Type":"ContainerStarted","Data":"ee93cc3889f2c41945ff930b4a80219f8b18be6445a32c5fa82c49ae608e9044"}
Sep 05 07:52:51.759638 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:52:51.759591    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-chbv7" podStartSLOduration=1.759576619 podStartE2EDuration="1.759576619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:52:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:52:51.759198114 +0000 UTC m=+252.704519922" watchObservedRunningTime="2026-09-05 07:52:51.759576619 +0000 UTC m=+252.704898424"
Sep 05 07:53:17.829216 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.829180    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"]
Sep 05 07:53:17.834652 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.834632    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.838403 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.838376    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 05 07:53:17.838550 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.838487    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 05 07:53:17.838550 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.838505    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 05 07:53:17.838816 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.838798    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mb5bz\""
Sep 05 07:53:17.839802 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.839787    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 05 07:53:17.868690 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.868654    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"]
Sep 05 07:53:17.886797 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.886767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.886970 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.886809    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.886970 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.886876    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-tmp\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.887060 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.886983    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn85h\" (UniqueName: \"kubernetes.io/projected/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-kube-api-access-gn85h\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.887060 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.887015    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-manifests\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.987419 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.987383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.987419 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.987434    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-tmp\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.987684 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.987484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn85h\" (UniqueName: \"kubernetes.io/projected/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-kube-api-access-gn85h\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.987684 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.987508    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-manifests\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.987684 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.987556    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.987997 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.987978    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-manifests\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.989961 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.989932    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-tmp\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.990181 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.990161    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:17.990245 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.990195    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:18.000818 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:17.997943    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn85h\" (UniqueName: \"kubernetes.io/projected/b0f0e25c-ce27-4b81-ad2a-0ee1622992c9-kube-api-access-gn85h\") pod \"opendatahub-operator-controller-manager-9fc4f56d7-rf9rp\" (UID: \"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:18.145806 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:18.145697    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:18.454141 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:18.454115    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"]
Sep 05 07:53:18.455023 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:53:18.455002    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f0e25c_ce27_4b81_ad2a_0ee1622992c9.slice/crio-c9838638d3800300501267d50be4a3b76481ec011e14f7e8743d7c6d668c67d3 WatchSource:0}: Error finding container c9838638d3800300501267d50be4a3b76481ec011e14f7e8743d7c6d668c67d3: Status 404 returned error can't find the container with id c9838638d3800300501267d50be4a3b76481ec011e14f7e8743d7c6d668c67d3
Sep 05 07:53:18.853174 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:18.853143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp" event={"ID":"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9","Type":"ContainerStarted","Data":"c9838638d3800300501267d50be4a3b76481ec011e14f7e8743d7c6d668c67d3"}
Sep 05 07:53:21.871459 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:21.871424    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp" event={"ID":"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9","Type":"ContainerStarted","Data":"c235bd8d1c063c7bed7995d4dd9e02b0edf120b052f2e3197f645232c8d1e27a"}
Sep 05 07:53:22.877784 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:22.877745    2569 generic.go:356] "Generic (PLEG): container finished" podID="b0f0e25c-ce27-4b81-ad2a-0ee1622992c9" containerID="c235bd8d1c063c7bed7995d4dd9e02b0edf120b052f2e3197f645232c8d1e27a" exitCode=0
Sep 05 07:53:22.878250 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:22.877791    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp" event={"ID":"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9","Type":"ContainerDied","Data":"c235bd8d1c063c7bed7995d4dd9e02b0edf120b052f2e3197f645232c8d1e27a"}
Sep 05 07:53:23.885370 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:23.885334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp" event={"ID":"b0f0e25c-ce27-4b81-ad2a-0ee1622992c9","Type":"ContainerStarted","Data":"c839848b7ef6c9f41c2aaf5dd223b7935c03fcfcf89f581d2d75e219fc532058"}
Sep 05 07:53:23.885832 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:23.885420    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:23.914511 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:23.914460    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp" podStartSLOduration=2.483165815 podStartE2EDuration="6.914444411s" totalImagesPullingTime="3.283438452s" totalInitContainerRuntime="1.147840144s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:53:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:53:23.912864551 +0000 UTC m=+284.858186356" watchObservedRunningTime="2026-09-05 07:53:23.914444411 +0000 UTC m=+284.859766215"
Sep 05 07:53:34.894267 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:34.894240    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9fc4f56d7-rf9rp"
Sep 05 07:53:39.493732 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:39.493698    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 07:53:39.496601 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:39.496579    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 07:53:39.500640 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:39.500619    2569 kubelet.go:1648] "Image garbage collection succeeded"
Sep 05 07:53:56.061300 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.061258    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd655598-ndghx"]
Sep 05 07:53:56.066161 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.066145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.072921 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.072892    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 05 07:53:56.073037 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.072954    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 05 07:53:56.073037 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.072957    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 05 07:53:56.073155 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.073115    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7gtqx\""
Sep 05 07:53:56.074339 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.074318    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 05 07:53:56.074455 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.074369    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 05 07:53:56.082341 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.082318    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd655598-ndghx"]
Sep 05 07:53:56.127338 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.127309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/70f0684c-52f2-43d0-ad16-8f358f2be4c4-metrics-cert\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.127453 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.127347    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/70f0684c-52f2-43d0-ad16-8f358f2be4c4-manager-config\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.127453 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.127432    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70f0684c-52f2-43d0-ad16-8f358f2be4c4-cert\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.127595 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.127467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg9sj\" (UniqueName: \"kubernetes.io/projected/70f0684c-52f2-43d0-ad16-8f358f2be4c4-kube-api-access-sg9sj\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.228911 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.228857    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70f0684c-52f2-43d0-ad16-8f358f2be4c4-cert\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.229128 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.228915    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg9sj\" (UniqueName: \"kubernetes.io/projected/70f0684c-52f2-43d0-ad16-8f358f2be4c4-kube-api-access-sg9sj\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.229128 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.228954    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/70f0684c-52f2-43d0-ad16-8f358f2be4c4-metrics-cert\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.229128 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.228994    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/70f0684c-52f2-43d0-ad16-8f358f2be4c4-manager-config\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.229919 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.229839    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/70f0684c-52f2-43d0-ad16-8f358f2be4c4-manager-config\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.231783 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.231756    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70f0684c-52f2-43d0-ad16-8f358f2be4c4-cert\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.231926 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.231858    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/70f0684c-52f2-43d0-ad16-8f358f2be4c4-metrics-cert\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.244039 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.244014    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg9sj\" (UniqueName: \"kubernetes.io/projected/70f0684c-52f2-43d0-ad16-8f358f2be4c4-kube-api-access-sg9sj\") pod \"lws-controller-manager-bd655598-ndghx\" (UID: \"70f0684c-52f2-43d0-ad16-8f358f2be4c4\") " pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.375694 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.375655    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:53:56.684608 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.684581    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd655598-ndghx"]
Sep 05 07:53:56.686753 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:53:56.686730    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f0684c_52f2_43d0_ad16_8f358f2be4c4.slice/crio-bd26135d4f260343981af6c28e8677c4d7baf42f0fa9b3808adcfc6e175a7cae WatchSource:0}: Error finding container bd26135d4f260343981af6c28e8677c4d7baf42f0fa9b3808adcfc6e175a7cae: Status 404 returned error can't find the container with id bd26135d4f260343981af6c28e8677c4d7baf42f0fa9b3808adcfc6e175a7cae
Sep 05 07:53:56.688779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:56.688721    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 07:53:57.037041 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:57.036960    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx" event={"ID":"70f0684c-52f2-43d0-ad16-8f358f2be4c4","Type":"ContainerStarted","Data":"bd26135d4f260343981af6c28e8677c4d7baf42f0fa9b3808adcfc6e175a7cae"}
Sep 05 07:53:58.855621 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.855590    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"]
Sep 05 07:53:58.869324 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.869294    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"
Sep 05 07:53:58.870377 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.870352    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"]
Sep 05 07:53:58.872734 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.872707    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 05 07:53:58.874192 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.874090    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4grxc\""
Sep 05 07:53:58.874748 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.874725    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"]
Sep 05 07:53:58.874852 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.874828    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"
Sep 05 07:53:58.877825 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.877799    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-t5dv8\""
Sep 05 07:53:58.879153 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.879133    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 05 07:53:58.894422 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.894393    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"]
Sep 05 07:53:58.954391 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.954355    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6189754b-abc7-4675-a676-6bb1babd4599-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tsc29\" (UID: \"6189754b-abc7-4675-a676-6bb1babd4599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"
Sep 05 07:53:58.954391 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.954409    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c16f8ac7-d8ed-4834-b3d0-c582cee5d746-cert\") pod \"kserve-controller-manager-d6f4b65b8-zw6gz\" (UID: \"c16f8ac7-d8ed-4834-b3d0-c582cee5d746\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"
Sep 05 07:53:58.954651 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.954472    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k672x\" (UniqueName: \"kubernetes.io/projected/6189754b-abc7-4675-a676-6bb1babd4599-kube-api-access-k672x\") pod \"llmisvc-controller-manager-777c6bd7d5-tsc29\" (UID: \"6189754b-abc7-4675-a676-6bb1babd4599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"
Sep 05 07:53:58.954651 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:58.954551    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57crq\" (UniqueName: \"kubernetes.io/projected/c16f8ac7-d8ed-4834-b3d0-c582cee5d746-kube-api-access-57crq\") pod \"kserve-controller-manager-d6f4b65b8-zw6gz\" (UID: \"c16f8ac7-d8ed-4834-b3d0-c582cee5d746\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"
Sep 05 07:53:59.055676 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.055646    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6189754b-abc7-4675-a676-6bb1babd4599-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tsc29\" (UID: \"6189754b-abc7-4675-a676-6bb1babd4599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"
Sep 05 07:53:59.055859 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.055700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c16f8ac7-d8ed-4834-b3d0-c582cee5d746-cert\") pod \"kserve-controller-manager-d6f4b65b8-zw6gz\" (UID: \"c16f8ac7-d8ed-4834-b3d0-c582cee5d746\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"
Sep 05 07:53:59.055859 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.055751    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k672x\" (UniqueName: \"kubernetes.io/projected/6189754b-abc7-4675-a676-6bb1babd4599-kube-api-access-k672x\") pod \"llmisvc-controller-manager-777c6bd7d5-tsc29\" (UID: \"6189754b-abc7-4675-a676-6bb1babd4599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"
Sep 05 07:53:59.055859 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.055820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57crq\" (UniqueName: \"kubernetes.io/projected/c16f8ac7-d8ed-4834-b3d0-c582cee5d746-kube-api-access-57crq\") pod \"kserve-controller-manager-d6f4b65b8-zw6gz\" (UID: \"c16f8ac7-d8ed-4834-b3d0-c582cee5d746\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"
Sep 05 07:53:59.058571 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.058543    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6189754b-abc7-4675-a676-6bb1babd4599-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tsc29\" (UID: \"6189754b-abc7-4675-a676-6bb1babd4599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"
Sep 05 07:53:59.058703 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.058609    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c16f8ac7-d8ed-4834-b3d0-c582cee5d746-cert\") pod \"kserve-controller-manager-d6f4b65b8-zw6gz\" (UID: \"c16f8ac7-d8ed-4834-b3d0-c582cee5d746\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"
Sep 05 07:53:59.065937 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.065893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k672x\" (UniqueName: \"kubernetes.io/projected/6189754b-abc7-4675-a676-6bb1babd4599-kube-api-access-k672x\") pod \"llmisvc-controller-manager-777c6bd7d5-tsc29\" (UID: \"6189754b-abc7-4675-a676-6bb1babd4599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"
Sep 05 07:53:59.066355 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.066324    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57crq\" (UniqueName: \"kubernetes.io/projected/c16f8ac7-d8ed-4834-b3d0-c582cee5d746-kube-api-access-57crq\") pod \"kserve-controller-manager-d6f4b65b8-zw6gz\" (UID: \"c16f8ac7-d8ed-4834-b3d0-c582cee5d746\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"
Sep 05 07:53:59.187106 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.187078    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"
Sep 05 07:53:59.195041 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.195020    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"
Sep 05 07:53:59.492178 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.492153    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"]
Sep 05 07:53:59.493979 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:53:59.493956    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc16f8ac7_d8ed_4834_b3d0_c582cee5d746.slice/crio-cf0dbea18333749146f7f2b1d006b0a13b036db978243735824adeae3ea50000 WatchSource:0}: Error finding container cf0dbea18333749146f7f2b1d006b0a13b036db978243735824adeae3ea50000: Status 404 returned error can't find the container with id cf0dbea18333749146f7f2b1d006b0a13b036db978243735824adeae3ea50000
Sep 05 07:53:59.511097 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:53:59.511078    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"]
Sep 05 07:53:59.513063 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:53:59.513038    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6189754b_abc7_4675_a676_6bb1babd4599.slice/crio-1032b620391ab0e80e5692124059054379390a7e7d22cf6efd79e9e8ebb6e983 WatchSource:0}: Error finding container 1032b620391ab0e80e5692124059054379390a7e7d22cf6efd79e9e8ebb6e983: Status 404 returned error can't find the container with id 1032b620391ab0e80e5692124059054379390a7e7d22cf6efd79e9e8ebb6e983
Sep 05 07:54:00.058187 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:00.057658    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx" event={"ID":"70f0684c-52f2-43d0-ad16-8f358f2be4c4","Type":"ContainerStarted","Data":"f8bb3721ed02dee0c5c588a0487c8faa1e89ac5469811b3f3ec85ee1167b4874"}
Sep 05 07:54:00.058691 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:00.058668    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:54:00.061129 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:00.061055    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz" event={"ID":"c16f8ac7-d8ed-4834-b3d0-c582cee5d746","Type":"ContainerStarted","Data":"cf0dbea18333749146f7f2b1d006b0a13b036db978243735824adeae3ea50000"}
Sep 05 07:54:00.064782 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:00.064757    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29" event={"ID":"6189754b-abc7-4675-a676-6bb1babd4599","Type":"ContainerStarted","Data":"1032b620391ab0e80e5692124059054379390a7e7d22cf6efd79e9e8ebb6e983"}
Sep 05 07:54:00.087037 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:00.086990    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx" podStartSLOduration=1.5904809 podStartE2EDuration="4.086976893s" totalImagesPullingTime="2.496495993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:54:00.086073093 +0000 UTC m=+321.031394900" watchObservedRunningTime="2026-09-05 07:54:00.086976893 +0000 UTC m=+321.032298751"
Sep 05 07:54:03.082190 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:03.082150    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29" event={"ID":"6189754b-abc7-4675-a676-6bb1babd4599","Type":"ContainerStarted","Data":"33e9162b490bafeeb3f3451bc36fa40aac5e27a5eeb6bf67f8af329390131e55"}
Sep 05 07:54:03.082578 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:03.082222    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"
Sep 05 07:54:03.083663 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:03.083642    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz" event={"ID":"c16f8ac7-d8ed-4834-b3d0-c582cee5d746","Type":"ContainerStarted","Data":"04e144f4d0887978e3891b5777c6630486949f9f9e2a16779f9b1d4f945995af"}
Sep 05 07:54:03.083731 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:03.083683    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"
Sep 05 07:54:03.122930 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:03.122868    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29" podStartSLOduration=1.652754999 podStartE2EDuration="5.122850562s" totalImagesPullingTime="3.470095563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:53:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:54:03.12196308 +0000 UTC m=+324.067284885" watchObservedRunningTime="2026-09-05 07:54:03.122850562 +0000 UTC m=+324.068172368"
Sep 05 07:54:03.154569 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:03.154503    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz" podStartSLOduration=1.717254072 podStartE2EDuration="5.154488013s" totalImagesPullingTime="3.437233941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:53:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:54:03.153547364 +0000 UTC m=+324.098869167" watchObservedRunningTime="2026-09-05 07:54:03.154488013 +0000 UTC m=+324.099809818"
Sep 05 07:54:12.077989 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:12.077957    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-bd655598-ndghx"
Sep 05 07:54:34.090967 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:34.090887    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tsc29"
Sep 05 07:54:34.093793 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:34.093775    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zw6gz"
Sep 05 07:54:36.522582 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.522556    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bbfb68d4f-2dt4n"]
Sep 05 07:54:36.526064 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.526043    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.538912 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.538887    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bbfb68d4f-2dt4n"]
Sep 05 07:54:36.599018 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.598982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-trusted-ca-bundle\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.599212 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.599041    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-console-oauth-config\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.599212 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.599075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-service-ca\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.599212 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.599119    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5q7d\" (UniqueName: \"kubernetes.io/projected/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-kube-api-access-c5q7d\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.599212 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.599170    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-console-serving-cert\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.599212 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.599211    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-oauth-serving-cert\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.599421 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.599242    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-console-config\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.699732 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.699705    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-console-config\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.699909 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.699744    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-trusted-ca-bundle\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.699992 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.699899    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-console-oauth-config\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.700059 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.699998    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-service-ca\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.700059 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.700023    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5q7d\" (UniqueName: \"kubernetes.io/projected/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-kube-api-access-c5q7d\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.700174 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.700058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-console-serving-cert\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.700174 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.700099    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-oauth-serving-cert\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.700584 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.700558    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-console-config\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.701399 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.701346    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-service-ca\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.701399 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.701408    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-oauth-serving-cert\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.701601 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.700670    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-trusted-ca-bundle\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.706852 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.706473    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-console-oauth-config\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.712688 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.712632    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-console-serving-cert\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.713762 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.713714    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5q7d\" (UniqueName: \"kubernetes.io/projected/ac2faf70-6059-45c1-b9f0-57251f8e9ef3-kube-api-access-c5q7d\") pod \"console-7bbfb68d4f-2dt4n\" (UID: \"ac2faf70-6059-45c1-b9f0-57251f8e9ef3\") " pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:36.835375 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:36.835342    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:37.106840 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:37.106816    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bbfb68d4f-2dt4n"]
Sep 05 07:54:37.108603 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:54:37.108579    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac2faf70_6059_45c1_b9f0_57251f8e9ef3.slice/crio-3040f32df3ca0ace86f85669d7b44ab9c0e0ab557fd2df27b20dcb9c7ae738cf WatchSource:0}: Error finding container 3040f32df3ca0ace86f85669d7b44ab9c0e0ab557fd2df27b20dcb9c7ae738cf: Status 404 returned error can't find the container with id 3040f32df3ca0ace86f85669d7b44ab9c0e0ab557fd2df27b20dcb9c7ae738cf
Sep 05 07:54:37.244963 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:37.244930    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bbfb68d4f-2dt4n" event={"ID":"ac2faf70-6059-45c1-b9f0-57251f8e9ef3","Type":"ContainerStarted","Data":"1db7dfe0862ce1954dcbb0d6e305b872d883a189ceb5b6e124a8cc9584bd39a3"}
Sep 05 07:54:37.245246 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:37.244979    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bbfb68d4f-2dt4n" event={"ID":"ac2faf70-6059-45c1-b9f0-57251f8e9ef3","Type":"ContainerStarted","Data":"3040f32df3ca0ace86f85669d7b44ab9c0e0ab557fd2df27b20dcb9c7ae738cf"}
Sep 05 07:54:37.265080 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:37.265028    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bbfb68d4f-2dt4n" podStartSLOduration=1.26501324 podStartE2EDuration="1.26501324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:54:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:54:37.263304487 +0000 UTC m=+358.208626286" watchObservedRunningTime="2026-09-05 07:54:37.26501324 +0000 UTC m=+358.210335045"
Sep 05 07:54:46.835807 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:46.835766    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:46.836309 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:46.835836    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:46.840461 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:46.840442    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:47.294633 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:47.294551    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bbfb68d4f-2dt4n"
Sep 05 07:54:47.341180 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:54:47.341142    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c486d8d79-ct8wh"]
Sep 05 07:55:12.370933 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.370872    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c486d8d79-ct8wh" podUID="4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" containerName="console" containerID="cri-o://ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514" gracePeriod=15
Sep 05 07:55:12.612883 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.612859    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:55:12.624485 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.624423    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkpgz\" (UniqueName: \"kubernetes.io/projected/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-kube-api-access-dkpgz\") pod \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") "
Sep 05 07:55:12.624485 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.624469    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-serving-cert\") pod \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") "
Sep 05 07:55:12.624661 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.624500    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-oauth-config\") pod \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") "
Sep 05 07:55:12.624661 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.624539    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-service-ca\") pod \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") "
Sep 05 07:55:12.624661 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.624564    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-trusted-ca-bundle\") pod \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") "
Sep 05 07:55:12.624661 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.624597    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-config\") pod \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") "
Sep 05 07:55:12.624854 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.624665    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-oauth-serving-cert\") pod \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\" (UID: \"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4\") "
Sep 05 07:55:12.624989 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.624961    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-service-ca" (OuterVolumeSpecName: "service-ca") pod "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" (UID: "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:55:12.625070 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.624985    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-config" (OuterVolumeSpecName: "console-config") pod "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" (UID: "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:55:12.625070 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.625013    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" (UID: "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:55:12.625213 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.625196    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" (UID: "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:55:12.626987 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.626965    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" (UID: "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:55:12.627260 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.627231    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" (UID: "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:55:12.627349 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.627251    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-kube-api-access-dkpgz" (OuterVolumeSpecName: "kube-api-access-dkpgz") pod "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" (UID: "4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4"). InnerVolumeSpecName "kube-api-access-dkpgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:55:12.725497 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.725462    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-oauth-serving-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:12.725497 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.725494    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkpgz\" (UniqueName: \"kubernetes.io/projected/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-kube-api-access-dkpgz\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:12.725497 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.725506    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-serving-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:12.725745 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.725514    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-oauth-config\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:12.725745 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.725543    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:12.725745 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.725552    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-trusted-ca-bundle\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:12.725745 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:12.725560    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4-console-config\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:13.319084 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.319056    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"]
Sep 05 07:55:13.319453 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.319439    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" containerName="console"
Sep 05 07:55:13.319497 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.319456    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" containerName="console"
Sep 05 07:55:13.319629 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.319610    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" containerName="console"
Sep 05 07:55:13.323179 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.323161    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:13.326294 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.326272    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 05 07:55:13.326294 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.326282    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dr9bw\""
Sep 05 07:55:13.326460 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.326278    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 05 07:55:13.329826 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.329803    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48fe1f9a-05d5-44fa-a11c-6e8812098599-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" (UID: \"48fe1f9a-05d5-44fa-a11c-6e8812098599\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:13.329944 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.329843    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdz6f\" (UniqueName: \"kubernetes.io/projected/48fe1f9a-05d5-44fa-a11c-6e8812098599-kube-api-access-gdz6f\") pod \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" (UID: \"48fe1f9a-05d5-44fa-a11c-6e8812098599\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:13.337741 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.337718    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"]
Sep 05 07:55:13.419916 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.419883    2569 generic.go:356] "Generic (PLEG): container finished" podID="4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" containerID="ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514" exitCode=0
Sep 05 07:55:13.420309 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.419925    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c486d8d79-ct8wh"
Sep 05 07:55:13.420309 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.419981    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c486d8d79-ct8wh" event={"ID":"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4","Type":"ContainerDied","Data":"ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514"}
Sep 05 07:55:13.420309 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.420030    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c486d8d79-ct8wh" event={"ID":"4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4","Type":"ContainerDied","Data":"70f3786493e858ea58ab35aaa67bcafaf0831f284f9bc60c39cd0391ed445e67"}
Sep 05 07:55:13.420309 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.420050    2569 scope.go:117] "RemoveContainer" containerID="ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514"
Sep 05 07:55:13.429297 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.429280    2569 scope.go:117] "RemoveContainer" containerID="ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514"
Sep 05 07:55:13.429638 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:13.429616    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514\": container with ID starting with ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514 not found: ID does not exist" containerID="ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514"
Sep 05 07:55:13.429717 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.429645    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514"} err="failed to get container status \"ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514\": rpc error: code = NotFound desc = could not find container \"ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514\": container with ID starting with ada73c5888ec6101e7ac246a7ed20d0bcffe307e6b1c8ec0cff71ae003fef514 not found: ID does not exist"
Sep 05 07:55:13.430984 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.430967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48fe1f9a-05d5-44fa-a11c-6e8812098599-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" (UID: \"48fe1f9a-05d5-44fa-a11c-6e8812098599\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:13.431028 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.430996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdz6f\" (UniqueName: \"kubernetes.io/projected/48fe1f9a-05d5-44fa-a11c-6e8812098599-kube-api-access-gdz6f\") pod \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" (UID: \"48fe1f9a-05d5-44fa-a11c-6e8812098599\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:13.431351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.431334    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48fe1f9a-05d5-44fa-a11c-6e8812098599-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" (UID: \"48fe1f9a-05d5-44fa-a11c-6e8812098599\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:13.441653 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.441634    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdz6f\" (UniqueName: \"kubernetes.io/projected/48fe1f9a-05d5-44fa-a11c-6e8812098599-kube-api-access-gdz6f\") pod \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" (UID: \"48fe1f9a-05d5-44fa-a11c-6e8812098599\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:13.447809 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.447788    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c486d8d79-ct8wh"]
Sep 05 07:55:13.449746 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.449726    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c486d8d79-ct8wh"]
Sep 05 07:55:13.570025 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.569952    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4" path="/var/lib/kubelet/pods/4d4ede85-6c3e-4b47-91ef-a0e6cf90c2d4/volumes"
Sep 05 07:55:13.634280 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.634255    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:13.912912 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:13.912888    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"]
Sep 05 07:55:13.915225 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:55:13.915198    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48fe1f9a_05d5_44fa_a11c_6e8812098599.slice/crio-5120c531f7fb8b506e7c20c7237df5a6bab0be813db11d29ac856ff566770fae WatchSource:0}: Error finding container 5120c531f7fb8b506e7c20c7237df5a6bab0be813db11d29ac856ff566770fae: Status 404 returned error can't find the container with id 5120c531f7fb8b506e7c20c7237df5a6bab0be813db11d29ac856ff566770fae
Sep 05 07:55:14.427256 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:14.427220    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t" event={"ID":"48fe1f9a-05d5-44fa-a11c-6e8812098599","Type":"ContainerStarted","Data":"5120c531f7fb8b506e7c20c7237df5a6bab0be813db11d29ac856ff566770fae"}
Sep 05 07:55:18.454592 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:18.454553    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t" event={"ID":"48fe1f9a-05d5-44fa-a11c-6e8812098599","Type":"ContainerStarted","Data":"07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3"}
Sep 05 07:55:18.454995 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:18.454676    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:18.476309 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:18.476260    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t" podStartSLOduration=1.5683436 podStartE2EDuration="5.476246355s" totalImagesPullingTime="3.907902755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:55:18.47456754 +0000 UTC m=+399.419889348" watchObservedRunningTime="2026-09-05 07:55:18.476246355 +0000 UTC m=+399.421568159"
Sep 05 07:55:29.462032 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:29.462001    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:30.861762 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:30.861724    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"]
Sep 05 07:55:30.875254 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:30.875223    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:30.879210 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:30.879188    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"]
Sep 05 07:55:30.987707 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:30.987676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wphn8\" (UniqueName: \"kubernetes.io/projected/5cc77dd9-fc9f-45a8-abae-30e01ec53458-kube-api-access-wphn8\") pod \"kuadrant-operator-controller-manager-96db98c47-zdnqn\" (UID: \"5cc77dd9-fc9f-45a8-abae-30e01ec53458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:30.987876 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:30.987740    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cc77dd9-fc9f-45a8-abae-30e01ec53458-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-zdnqn\" (UID: \"5cc77dd9-fc9f-45a8-abae-30e01ec53458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:30.992155 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:30.992131    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"]
Sep 05 07:55:30.992371 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:30.992352    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-wphn8], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn" podUID="5cc77dd9-fc9f-45a8-abae-30e01ec53458"
Sep 05 07:55:31.077706 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.077674    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"]
Sep 05 07:55:31.081117 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.081098    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:31.088101 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.088078    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wphn8\" (UniqueName: \"kubernetes.io/projected/5cc77dd9-fc9f-45a8-abae-30e01ec53458-kube-api-access-wphn8\") pod \"kuadrant-operator-controller-manager-96db98c47-zdnqn\" (UID: \"5cc77dd9-fc9f-45a8-abae-30e01ec53458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:31.088180 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.088137    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cc77dd9-fc9f-45a8-abae-30e01ec53458-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-zdnqn\" (UID: \"5cc77dd9-fc9f-45a8-abae-30e01ec53458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:31.088622 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.088602    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cc77dd9-fc9f-45a8-abae-30e01ec53458-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-zdnqn\" (UID: \"5cc77dd9-fc9f-45a8-abae-30e01ec53458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:31.112198 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.112143    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wphn8\" (UniqueName: \"kubernetes.io/projected/5cc77dd9-fc9f-45a8-abae-30e01ec53458-kube-api-access-wphn8\") pod \"kuadrant-operator-controller-manager-96db98c47-zdnqn\" (UID: \"5cc77dd9-fc9f-45a8-abae-30e01ec53458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:31.115323 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.115298    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"]
Sep 05 07:55:31.189438 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.189411    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/161f665e-4a24-4b59-935f-bdabfbd63497-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" (UID: \"161f665e-4a24-4b59-935f-bdabfbd63497\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:31.189577 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.189454    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqrjd\" (UniqueName: \"kubernetes.io/projected/161f665e-4a24-4b59-935f-bdabfbd63497-kube-api-access-dqrjd\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" (UID: \"161f665e-4a24-4b59-935f-bdabfbd63497\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:31.290197 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.290170    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/161f665e-4a24-4b59-935f-bdabfbd63497-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" (UID: \"161f665e-4a24-4b59-935f-bdabfbd63497\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:31.290315 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.290217    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqrjd\" (UniqueName: \"kubernetes.io/projected/161f665e-4a24-4b59-935f-bdabfbd63497-kube-api-access-dqrjd\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" (UID: \"161f665e-4a24-4b59-935f-bdabfbd63497\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:31.290615 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.290597    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/161f665e-4a24-4b59-935f-bdabfbd63497-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" (UID: \"161f665e-4a24-4b59-935f-bdabfbd63497\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:31.299484 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.299463    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqrjd\" (UniqueName: \"kubernetes.io/projected/161f665e-4a24-4b59-935f-bdabfbd63497-kube-api-access-dqrjd\") pod \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" (UID: \"161f665e-4a24-4b59-935f-bdabfbd63497\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:31.391253 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.391191    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:31.518224 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.518194    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:31.525289 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.525267    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:31.593193 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.593163    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wphn8\" (UniqueName: \"kubernetes.io/projected/5cc77dd9-fc9f-45a8-abae-30e01ec53458-kube-api-access-wphn8\") pod \"5cc77dd9-fc9f-45a8-abae-30e01ec53458\" (UID: \"5cc77dd9-fc9f-45a8-abae-30e01ec53458\") "
Sep 05 07:55:31.593348 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.593207    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cc77dd9-fc9f-45a8-abae-30e01ec53458-extensions-socket-volume\") pod \"5cc77dd9-fc9f-45a8-abae-30e01ec53458\" (UID: \"5cc77dd9-fc9f-45a8-abae-30e01ec53458\") "
Sep 05 07:55:31.593519 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.593500    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc77dd9-fc9f-45a8-abae-30e01ec53458-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5cc77dd9-fc9f-45a8-abae-30e01ec53458" (UID: "5cc77dd9-fc9f-45a8-abae-30e01ec53458"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:55:31.595446 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.595421    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cc77dd9-fc9f-45a8-abae-30e01ec53458-kube-api-access-wphn8" (OuterVolumeSpecName: "kube-api-access-wphn8") pod "5cc77dd9-fc9f-45a8-abae-30e01ec53458" (UID: "5cc77dd9-fc9f-45a8-abae-30e01ec53458"). InnerVolumeSpecName "kube-api-access-wphn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:55:31.670067 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.670042    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"]
Sep 05 07:55:31.672920 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:55:31.672896    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161f665e_4a24_4b59_935f_bdabfbd63497.slice/crio-a84a19f6e49ca3e717ecf1e3eb9b6cfad64bb633462c1b86eeac2e002f07403f WatchSource:0}: Error finding container a84a19f6e49ca3e717ecf1e3eb9b6cfad64bb633462c1b86eeac2e002f07403f: Status 404 returned error can't find the container with id a84a19f6e49ca3e717ecf1e3eb9b6cfad64bb633462c1b86eeac2e002f07403f
Sep 05 07:55:31.694132 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.694108    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wphn8\" (UniqueName: \"kubernetes.io/projected/5cc77dd9-fc9f-45a8-abae-30e01ec53458-kube-api-access-wphn8\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:31.694132 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.694130    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cc77dd9-fc9f-45a8-abae-30e01ec53458-extensions-socket-volume\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:31.887588 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.887547    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"]
Sep 05 07:55:31.888058 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.887783    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" containerName="manager" containerID="cri-o://07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3" gracePeriod=2
Sep 05 07:55:31.903829 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.903805    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"]
Sep 05 07:55:31.911796 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:31.911724    2569 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"48fe1f9a-05d5-44fa-a11c-6e8812098599\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-twf4t\": pods \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:31.916835 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.916794    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"]
Sep 05 07:55:31.919600 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.919515    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"]
Sep 05 07:55:31.919999 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.919979    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" containerName="manager"
Sep 05 07:55:31.919999 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.919996    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" containerName="manager"
Sep 05 07:55:31.920147 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.920091    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" containerName="manager"
Sep 05 07:55:31.923388 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.923368    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"]
Sep 05 07:55:31.923499 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.923478    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:31.926641 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:31.926615    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:31.941402 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.941353    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"]
Sep 05 07:55:31.944181 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.944105    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"]
Sep 05 07:55:31.957948 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.957918    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"]
Sep 05 07:55:31.997426 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.997396    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d89k8\" (UniqueName: \"kubernetes.io/projected/52489c8f-8bb1-4f71-925a-3610bb6928d7-kube-api-access-d89k8\") pod \"kuadrant-operator-controller-manager-58fc56757c-rc2mg\" (UID: \"52489c8f-8bb1-4f71-925a-3610bb6928d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:31.997654 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:31.997503    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52489c8f-8bb1-4f71-925a-3610bb6928d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-rc2mg\" (UID: \"52489c8f-8bb1-4f71-925a-3610bb6928d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:32.016968 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.016943    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f"]
Sep 05 07:55:32.017351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.017332    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" containerName="manager"
Sep 05 07:55:32.017351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.017348    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" containerName="manager"
Sep 05 07:55:32.017520 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.017444    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" containerName="manager"
Sep 05 07:55:32.020508 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.020489    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f"
Sep 05 07:55:32.023727 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.023692    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:32.024253 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.024230    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-d9xcp\""
Sep 05 07:55:32.032675 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.032652    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f"]
Sep 05 07:55:32.099142 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.099111    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d89k8\" (UniqueName: \"kubernetes.io/projected/52489c8f-8bb1-4f71-925a-3610bb6928d7-kube-api-access-d89k8\") pod \"kuadrant-operator-controller-manager-58fc56757c-rc2mg\" (UID: \"52489c8f-8bb1-4f71-925a-3610bb6928d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:32.099312 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.099202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52489c8f-8bb1-4f71-925a-3610bb6928d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-rc2mg\" (UID: \"52489c8f-8bb1-4f71-925a-3610bb6928d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:32.099312 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.099239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhdbs\" (UniqueName: \"kubernetes.io/projected/ecabc342-f49c-4e5a-a6b2-663d2d60bf8b-kube-api-access-nhdbs\") pod \"limitador-operator-controller-manager-6c549b6d85-rl65f\" (UID: \"ecabc342-f49c-4e5a-a6b2-663d2d60bf8b\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f"
Sep 05 07:55:32.099632 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.099610    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52489c8f-8bb1-4f71-925a-3610bb6928d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-rc2mg\" (UID: \"52489c8f-8bb1-4f71-925a-3610bb6928d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:32.111105 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.111079    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d89k8\" (UniqueName: \"kubernetes.io/projected/52489c8f-8bb1-4f71-925a-3610bb6928d7-kube-api-access-d89k8\") pod \"kuadrant-operator-controller-manager-58fc56757c-rc2mg\" (UID: \"52489c8f-8bb1-4f71-925a-3610bb6928d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:32.137563 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.137520    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:32.140372 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.140338    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:32.199894 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.199799    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdz6f\" (UniqueName: \"kubernetes.io/projected/48fe1f9a-05d5-44fa-a11c-6e8812098599-kube-api-access-gdz6f\") pod \"48fe1f9a-05d5-44fa-a11c-6e8812098599\" (UID: \"48fe1f9a-05d5-44fa-a11c-6e8812098599\") "
Sep 05 07:55:32.199894 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.199864    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48fe1f9a-05d5-44fa-a11c-6e8812098599-extensions-socket-volume\") pod \"48fe1f9a-05d5-44fa-a11c-6e8812098599\" (UID: \"48fe1f9a-05d5-44fa-a11c-6e8812098599\") "
Sep 05 07:55:32.200102 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.200084    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhdbs\" (UniqueName: \"kubernetes.io/projected/ecabc342-f49c-4e5a-a6b2-663d2d60bf8b-kube-api-access-nhdbs\") pod \"limitador-operator-controller-manager-6c549b6d85-rl65f\" (UID: \"ecabc342-f49c-4e5a-a6b2-663d2d60bf8b\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f"
Sep 05 07:55:32.200474 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.200447    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48fe1f9a-05d5-44fa-a11c-6e8812098599-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "48fe1f9a-05d5-44fa-a11c-6e8812098599" (UID: "48fe1f9a-05d5-44fa-a11c-6e8812098599"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:55:32.202313 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.202289    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48fe1f9a-05d5-44fa-a11c-6e8812098599-kube-api-access-gdz6f" (OuterVolumeSpecName: "kube-api-access-gdz6f") pod "48fe1f9a-05d5-44fa-a11c-6e8812098599" (UID: "48fe1f9a-05d5-44fa-a11c-6e8812098599"). InnerVolumeSpecName "kube-api-access-gdz6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:55:32.215443 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.215422    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhdbs\" (UniqueName: \"kubernetes.io/projected/ecabc342-f49c-4e5a-a6b2-663d2d60bf8b-kube-api-access-nhdbs\") pod \"limitador-operator-controller-manager-6c549b6d85-rl65f\" (UID: \"ecabc342-f49c-4e5a-a6b2-663d2d60bf8b\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f"
Sep 05 07:55:32.270351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.270306    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:32.300684 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.300656    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdz6f\" (UniqueName: \"kubernetes.io/projected/48fe1f9a-05d5-44fa-a11c-6e8812098599-kube-api-access-gdz6f\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:32.300684 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.300682    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48fe1f9a-05d5-44fa-a11c-6e8812098599-extensions-socket-volume\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:32.335716 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.335687    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f"
Sep 05 07:55:32.525107 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.525038    2569 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5" container="manager"
Sep 05 07:55:32.526417 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.526393    2569 generic.go:356] "Generic (PLEG): container finished" podID="48fe1f9a-05d5-44fa-a11c-6e8812098599" containerID="07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3" exitCode=0
Sep 05 07:55:32.526565 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.526456    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:32.526565 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.526492    2569 scope.go:117] "RemoveContainer" containerID="07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3"
Sep 05 07:55:32.526659 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.526644    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:32.530664 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.530635    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:32.533456 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.533431    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:32.535726 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.535708    2569 scope.go:117] "RemoveContainer" containerID="07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3"
Sep 05 07:55:32.535834 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.535810    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-zdnqn\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="5cc77dd9-fc9f-45a8-abae-30e01ec53458" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:32.536031 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.536014    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3\": container with ID starting with 07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3 not found: ID does not exist" containerID="07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3"
Sep 05 07:55:32.536075 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.536048    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3"} err="failed to get container status \"07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3\": rpc error: code = NotFound desc = could not find container \"07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3\": container with ID starting with 07e81a6005c147b11d9ec77dfe1b50893fe67967000c4e9e4d6fd93905ef5ce3 not found: ID does not exist"
Sep 05 07:55:32.538021 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.537994    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:32.541208 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.541188    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:32.543710 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.543683    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:32.546154 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.546128    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:32.549485 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:32.549454    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-zdnqn\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="5cc77dd9-fc9f-45a8-abae-30e01ec53458" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:32.552479 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.552418    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"]
Sep 05 07:55:32.555186 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:55:32.555162    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52489c8f_8bb1_4f71_925a_3610bb6928d7.slice/crio-740006e42f808ea378af250844e2ab15cc29bcfe1745c115e4edd82b45bb7fd7 WatchSource:0}: Error finding container 740006e42f808ea378af250844e2ab15cc29bcfe1745c115e4edd82b45bb7fd7: Status 404 returned error can't find the container with id 740006e42f808ea378af250844e2ab15cc29bcfe1745c115e4edd82b45bb7fd7
Sep 05 07:55:32.614868 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:32.614843    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f"]
Sep 05 07:55:32.622565 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:55:32.622516    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecabc342_f49c_4e5a_a6b2_663d2d60bf8b.slice/crio-4b755bda4b49b7c357fdfeaf4e01c74d7975aebd90145c8b2fbcdd1cacd783fd WatchSource:0}: Error finding container 4b755bda4b49b7c357fdfeaf4e01c74d7975aebd90145c8b2fbcdd1cacd783fd: Status 404 returned error can't find the container with id 4b755bda4b49b7c357fdfeaf4e01c74d7975aebd90145c8b2fbcdd1cacd783fd
Sep 05 07:55:33.533835 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:33.533796    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg" event={"ID":"52489c8f-8bb1-4f71-925a-3610bb6928d7","Type":"ContainerStarted","Data":"3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb"}
Sep 05 07:55:33.534300 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:33.533889    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg" event={"ID":"52489c8f-8bb1-4f71-925a-3610bb6928d7","Type":"ContainerStarted","Data":"740006e42f808ea378af250844e2ab15cc29bcfe1745c115e4edd82b45bb7fd7"}
Sep 05 07:55:33.534300 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:33.533924    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:33.535704 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:33.535677    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" containerName="manager" containerID="cri-o://a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865" gracePeriod=2
Sep 05 07:55:33.536388 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:33.536359    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-twf4t\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-twf4t"
Sep 05 07:55:33.537175 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:33.537155    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f" event={"ID":"ecabc342-f49c-4e5a-a6b2-663d2d60bf8b","Type":"ContainerStarted","Data":"4b755bda4b49b7c357fdfeaf4e01c74d7975aebd90145c8b2fbcdd1cacd783fd"}
Sep 05 07:55:33.538660 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:33.538598    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:33.570544 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:33.570499    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48fe1f9a-05d5-44fa-a11c-6e8812098599" path="/var/lib/kubelet/pods/48fe1f9a-05d5-44fa-a11c-6e8812098599/volumes"
Sep 05 07:55:33.570916 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:33.570902    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cc77dd9-fc9f-45a8-abae-30e01ec53458" path="/var/lib/kubelet/pods/5cc77dd9-fc9f-45a8-abae-30e01ec53458/volumes"
Sep 05 07:55:33.579712 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:33.579657    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg" podStartSLOduration=2.579638457 podStartE2EDuration="2.579638457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:55:33.578349235 +0000 UTC m=+414.523671049" watchObservedRunningTime="2026-09-05 07:55:33.579638457 +0000 UTC m=+414.524960263"
Sep 05 07:55:33.580784 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:33.580755    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-zdnqn\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="5cc77dd9-fc9f-45a8-abae-30e01ec53458" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-zdnqn"
Sep 05 07:55:34.119236 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.119209    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:34.218096 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.218017    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/161f665e-4a24-4b59-935f-bdabfbd63497-extensions-socket-volume\") pod \"161f665e-4a24-4b59-935f-bdabfbd63497\" (UID: \"161f665e-4a24-4b59-935f-bdabfbd63497\") "
Sep 05 07:55:34.218096 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.218074    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqrjd\" (UniqueName: \"kubernetes.io/projected/161f665e-4a24-4b59-935f-bdabfbd63497-kube-api-access-dqrjd\") pod \"161f665e-4a24-4b59-935f-bdabfbd63497\" (UID: \"161f665e-4a24-4b59-935f-bdabfbd63497\") "
Sep 05 07:55:34.218377 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.218341    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/161f665e-4a24-4b59-935f-bdabfbd63497-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "161f665e-4a24-4b59-935f-bdabfbd63497" (UID: "161f665e-4a24-4b59-935f-bdabfbd63497"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:55:34.220284 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.220261    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/161f665e-4a24-4b59-935f-bdabfbd63497-kube-api-access-dqrjd" (OuterVolumeSpecName: "kube-api-access-dqrjd") pod "161f665e-4a24-4b59-935f-bdabfbd63497" (UID: "161f665e-4a24-4b59-935f-bdabfbd63497"). InnerVolumeSpecName "kube-api-access-dqrjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:55:34.318970 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.318936    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/161f665e-4a24-4b59-935f-bdabfbd63497-extensions-socket-volume\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:34.318970 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.318963    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqrjd\" (UniqueName: \"kubernetes.io/projected/161f665e-4a24-4b59-935f-bdabfbd63497-kube-api-access-dqrjd\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:34.545265 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.545238    2569 generic.go:356] "Generic (PLEG): container finished" podID="161f665e-4a24-4b59-935f-bdabfbd63497" containerID="a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865" exitCode=0
Sep 05 07:55:34.545604 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.545280    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:34.545604 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.545333    2569 scope.go:117] "RemoveContainer" containerID="a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865"
Sep 05 07:55:34.547437 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.547410    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f" event={"ID":"ecabc342-f49c-4e5a-a6b2-663d2d60bf8b","Type":"ContainerStarted","Data":"7bb35a797751307ef51fa2de5e87997de2ed0e7cebd4d5e5b5242eb6ae2d2ee8"}
Sep 05 07:55:34.548187 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:34.548161    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:34.555194 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.555177    2569 scope.go:117] "RemoveContainer" containerID="a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865"
Sep 05 07:55:34.555495 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:34.555473    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865\": container with ID starting with a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865 not found: ID does not exist" containerID="a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865"
Sep 05 07:55:34.555605 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:34.555501    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865"} err="failed to get container status \"a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865\": rpc error: code = NotFound desc = could not find container \"a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865\": container with ID starting with a23a73de11b122db58044e97347ff22f80727bb7c319c787f50cf7dc98c04865 not found: ID does not exist"
Sep 05 07:55:34.556198 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:34.556176    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:35.553181 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:35.553149    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f"
Sep 05 07:55:35.555792 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:35.555765    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-tp4p5\" is forbidden: User \"system:node:ip-10-0-140-64.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-64.ec2.internal' and this object" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-tp4p5"
Sep 05 07:55:35.569930 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:35.569905    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="161f665e-4a24-4b59-935f-bdabfbd63497" path="/var/lib/kubelet/pods/161f665e-4a24-4b59-935f-bdabfbd63497/volumes"
Sep 05 07:55:35.572960 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:35.572916    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f" podStartSLOduration=2.7798705679999998 podStartE2EDuration="4.572903167s" totalImagesPullingTime="1.793032599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:55:35.571758541 +0000 UTC m=+416.517080369" watchObservedRunningTime="2026-09-05 07:55:35.572903167 +0000 UTC m=+416.518225025"
Sep 05 07:55:44.550411 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:44.550372    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:46.560779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:46.560749    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-rl65f"
Sep 05 07:55:48.040600 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.040573    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"]
Sep 05 07:55:48.041011 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.040757    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg" podUID="52489c8f-8bb1-4f71-925a-3610bb6928d7" containerName="manager" containerID="cri-o://3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb" gracePeriod=10
Sep 05 07:55:48.287790 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.287761    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:48.449818 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.449793    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d89k8\" (UniqueName: \"kubernetes.io/projected/52489c8f-8bb1-4f71-925a-3610bb6928d7-kube-api-access-d89k8\") pod \"52489c8f-8bb1-4f71-925a-3610bb6928d7\" (UID: \"52489c8f-8bb1-4f71-925a-3610bb6928d7\") "
Sep 05 07:55:48.449968 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.449905    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52489c8f-8bb1-4f71-925a-3610bb6928d7-extensions-socket-volume\") pod \"52489c8f-8bb1-4f71-925a-3610bb6928d7\" (UID: \"52489c8f-8bb1-4f71-925a-3610bb6928d7\") "
Sep 05 07:55:48.450254 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.450232    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52489c8f-8bb1-4f71-925a-3610bb6928d7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "52489c8f-8bb1-4f71-925a-3610bb6928d7" (UID: "52489c8f-8bb1-4f71-925a-3610bb6928d7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:55:48.452046 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.452023    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52489c8f-8bb1-4f71-925a-3610bb6928d7-kube-api-access-d89k8" (OuterVolumeSpecName: "kube-api-access-d89k8") pod "52489c8f-8bb1-4f71-925a-3610bb6928d7" (UID: "52489c8f-8bb1-4f71-925a-3610bb6928d7"). InnerVolumeSpecName "kube-api-access-d89k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:55:48.550617 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.550592    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52489c8f-8bb1-4f71-925a-3610bb6928d7-extensions-socket-volume\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:48.550617 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.550612    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d89k8\" (UniqueName: \"kubernetes.io/projected/52489c8f-8bb1-4f71-925a-3610bb6928d7-kube-api-access-d89k8\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:55:48.622733 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.622700    2569 generic.go:356] "Generic (PLEG): container finished" podID="52489c8f-8bb1-4f71-925a-3610bb6928d7" containerID="3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb" exitCode=0
Sep 05 07:55:48.622873 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.622746    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"
Sep 05 07:55:48.622873 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.622775    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg" event={"ID":"52489c8f-8bb1-4f71-925a-3610bb6928d7","Type":"ContainerDied","Data":"3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb"}
Sep 05 07:55:48.622873 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.622814    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg" event={"ID":"52489c8f-8bb1-4f71-925a-3610bb6928d7","Type":"ContainerDied","Data":"740006e42f808ea378af250844e2ab15cc29bcfe1745c115e4edd82b45bb7fd7"}
Sep 05 07:55:48.622873 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.622835    2569 scope.go:117] "RemoveContainer" containerID="3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb"
Sep 05 07:55:48.631776 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.631757    2569 scope.go:117] "RemoveContainer" containerID="3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb"
Sep 05 07:55:48.632019 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:55:48.632001    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb\": container with ID starting with 3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb not found: ID does not exist" containerID="3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb"
Sep 05 07:55:48.632083 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.632030    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb"} err="failed to get container status \"3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb\": rpc error: code = NotFound desc = could not find container \"3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb\": container with ID starting with 3b9a8eff36f49470545f60c2581273c947c700238d365e0285c88c2f225bfbfb not found: ID does not exist"
Sep 05 07:55:48.649936 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.649913    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"]
Sep 05 07:55:48.660279 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:48.660260    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-rc2mg"]
Sep 05 07:55:49.575518 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:55:49.575476    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52489c8f-8bb1-4f71-925a-3610bb6928d7" path="/var/lib/kubelet/pods/52489c8f-8bb1-4f71-925a-3610bb6928d7/volumes"
Sep 05 07:56:04.565664 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.565582    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"]
Sep 05 07:56:04.566108 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.566084    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52489c8f-8bb1-4f71-925a-3610bb6928d7" containerName="manager"
Sep 05 07:56:04.566238 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.566223    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="52489c8f-8bb1-4f71-925a-3610bb6928d7" containerName="manager"
Sep 05 07:56:04.566435 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.566406    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="52489c8f-8bb1-4f71-925a-3610bb6928d7" containerName="manager"
Sep 05 07:56:04.569473 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.569456    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.572418 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.572388    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 05 07:56:04.572551 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.572426    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 05 07:56:04.572551 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.572388    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 05 07:56:04.572680 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.572616    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-w2x6f\""
Sep 05 07:56:04.581140 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.581112    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.581263 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.581157    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.581263 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.581183    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.581263 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.581210    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.581263 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.581256    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b378fb5-4133-490b-af8f-065518cc9b0a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.581510 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.581290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vsvt\" (UniqueName: \"kubernetes.io/projected/2b378fb5-4133-490b-af8f-065518cc9b0a-kube-api-access-8vsvt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.581510 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.581330    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.581510 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.581419    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.581510 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.581469    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.582842 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.582822    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"]
Sep 05 07:56:04.682001 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.681968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682201 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682007    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682201 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682023    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682201 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682040    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682201 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b378fb5-4133-490b-af8f-065518cc9b0a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682201 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682183    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vsvt\" (UniqueName: \"kubernetes.io/projected/2b378fb5-4133-490b-af8f-065518cc9b0a-kube-api-access-8vsvt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682422 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682241    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682422 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682422 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682327    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682612 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682442    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682624    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682654    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.682926 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.682839    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.683113 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.683038    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b378fb5-4133-490b-af8f-065518cc9b0a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.684596 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.684573    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.684821 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.684803    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.690480 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.690459    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b378fb5-4133-490b-af8f-065518cc9b0a-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.690677 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.690661    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vsvt\" (UniqueName: \"kubernetes.io/projected/2b378fb5-4133-490b-af8f-065518cc9b0a-kube-api-access-8vsvt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-bzs86\" (UID: \"2b378fb5-4133-490b-af8f-065518cc9b0a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:04.881190 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:04.881157    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:05.158544 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:05.158443    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"]
Sep 05 07:56:05.161954 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:56:05.161927    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b378fb5_4133_490b_af8f_065518cc9b0a.slice/crio-56b42fe19cb13f92c6b308cc16a1c2aebb4690e137ccb5603f73b0dc51ecc8c5 WatchSource:0}: Error finding container 56b42fe19cb13f92c6b308cc16a1c2aebb4690e137ccb5603f73b0dc51ecc8c5: Status 404 returned error can't find the container with id 56b42fe19cb13f92c6b308cc16a1c2aebb4690e137ccb5603f73b0dc51ecc8c5
Sep 05 07:56:05.708083 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:05.708046    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86" event={"ID":"2b378fb5-4133-490b-af8f-065518cc9b0a","Type":"ContainerStarted","Data":"56b42fe19cb13f92c6b308cc16a1c2aebb4690e137ccb5603f73b0dc51ecc8c5"}
Sep 05 07:56:07.795338 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:07.795301    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 07:56:07.795636 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:07.795380    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 07:56:07.795636 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:07.795409    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 07:56:08.726334 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:08.726295    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86" event={"ID":"2b378fb5-4133-490b-af8f-065518cc9b0a","Type":"ContainerStarted","Data":"846b2cf9554e564668c155db27be4a5cd7e1d473ea793a70505d30305964aff1"}
Sep 05 07:56:08.751565 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:08.751498    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86" podStartSLOduration=2.120567127 podStartE2EDuration="4.751483095s" totalImagesPullingTime="2.630915968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:56:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:56:08.74955286 +0000 UTC m=+449.694874662" watchObservedRunningTime="2026-09-05 07:56:08.751483095 +0000 UTC m=+449.696804899"
Sep 05 07:56:08.882262 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:08.882228    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:08.887128 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:08.887098    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:09.731047 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:09.731009    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:09.731911 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:09.731890    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:56:19.413030 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.412997    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"]
Sep 05 07:56:19.417898 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.417870    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:19.421763 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.421739    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t92sv\""
Sep 05 07:56:19.421947 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.421812    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 05 07:56:19.423725 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.423705    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"]
Sep 05 07:56:19.506514 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.506482    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"]
Sep 05 07:56:19.513734 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.513706    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-config-file\") pod \"limitador-limitador-859c4d96d6-bmgwv\" (UID: \"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:19.513892 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.513753    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dsnw\" (UniqueName: \"kubernetes.io/projected/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-kube-api-access-4dsnw\") pod \"limitador-limitador-859c4d96d6-bmgwv\" (UID: \"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:19.615071 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.615044    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-config-file\") pod \"limitador-limitador-859c4d96d6-bmgwv\" (UID: \"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:19.615244 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.615090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dsnw\" (UniqueName: \"kubernetes.io/projected/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-kube-api-access-4dsnw\") pod \"limitador-limitador-859c4d96d6-bmgwv\" (UID: \"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:19.615785 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.615767    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-config-file\") pod \"limitador-limitador-859c4d96d6-bmgwv\" (UID: \"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:19.625467 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.625438    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dsnw\" (UniqueName: \"kubernetes.io/projected/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-kube-api-access-4dsnw\") pod \"limitador-limitador-859c4d96d6-bmgwv\" (UID: \"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:19.731351 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:19.731275    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:20.011406 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.011335    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"]
Sep 05 07:56:20.014650 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:56:20.014619    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c7e2375_3ca0_4cb5_a78a_e9d335fbcb52.slice/crio-26140fa51c73e8af679a5e276f868f986403a5c3f684540217c49d660c991a68 WatchSource:0}: Error finding container 26140fa51c73e8af679a5e276f868f986403a5c3f684540217c49d660c991a68: Status 404 returned error can't find the container with id 26140fa51c73e8af679a5e276f868f986403a5c3f684540217c49d660c991a68
Sep 05 07:56:20.394839 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.394805    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-dpgb5"]
Sep 05 07:56:20.399445 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.399425    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-dpgb5"
Sep 05 07:56:20.402305 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.402276    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c2j9p\""
Sep 05 07:56:20.405237 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.405215    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-dpgb5"]
Sep 05 07:56:20.524812 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.524779    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nx45\" (UniqueName: \"kubernetes.io/projected/4c1bb145-37fa-4398-b3ce-507369fb22cf-kube-api-access-6nx45\") pod \"authorino-c548b5d7d-dpgb5\" (UID: \"4c1bb145-37fa-4398-b3ce-507369fb22cf\") " pod="kuadrant-system/authorino-c548b5d7d-dpgb5"
Sep 05 07:56:20.626137 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.626104    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nx45\" (UniqueName: \"kubernetes.io/projected/4c1bb145-37fa-4398-b3ce-507369fb22cf-kube-api-access-6nx45\") pod \"authorino-c548b5d7d-dpgb5\" (UID: \"4c1bb145-37fa-4398-b3ce-507369fb22cf\") " pod="kuadrant-system/authorino-c548b5d7d-dpgb5"
Sep 05 07:56:20.637267 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.637243    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nx45\" (UniqueName: \"kubernetes.io/projected/4c1bb145-37fa-4398-b3ce-507369fb22cf-kube-api-access-6nx45\") pod \"authorino-c548b5d7d-dpgb5\" (UID: \"4c1bb145-37fa-4398-b3ce-507369fb22cf\") " pod="kuadrant-system/authorino-c548b5d7d-dpgb5"
Sep 05 07:56:20.710084 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.710033    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-dpgb5"
Sep 05 07:56:20.791933 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.791893    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv" event={"ID":"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52","Type":"ContainerStarted","Data":"26140fa51c73e8af679a5e276f868f986403a5c3f684540217c49d660c991a68"}
Sep 05 07:56:20.988630 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:20.988605    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-dpgb5"]
Sep 05 07:56:21.142310 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:56:21.142281    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c1bb145_37fa_4398_b3ce_507369fb22cf.slice/crio-4408d943717681dd8e2adbb68a64ab4c86df5d5c85cd4e01a868bdbbdedae5ce WatchSource:0}: Error finding container 4408d943717681dd8e2adbb68a64ab4c86df5d5c85cd4e01a868bdbbdedae5ce: Status 404 returned error can't find the container with id 4408d943717681dd8e2adbb68a64ab4c86df5d5c85cd4e01a868bdbbdedae5ce
Sep 05 07:56:21.801784 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:21.801749    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv" event={"ID":"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52","Type":"ContainerStarted","Data":"64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6"}
Sep 05 07:56:21.802350 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:21.801835    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:21.803864 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:21.803818    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-dpgb5" event={"ID":"4c1bb145-37fa-4398-b3ce-507369fb22cf","Type":"ContainerStarted","Data":"4408d943717681dd8e2adbb68a64ab4c86df5d5c85cd4e01a868bdbbdedae5ce"}
Sep 05 07:56:21.820412 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:21.819964    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv" podStartSLOduration=1.638658199 podStartE2EDuration="2.81995205s" totalImagesPullingTime="1.181293851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:56:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:56:21.818953049 +0000 UTC m=+462.764274854" watchObservedRunningTime="2026-09-05 07:56:21.81995205 +0000 UTC m=+462.765273833"
Sep 05 07:56:23.817385 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:23.817345    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-dpgb5" event={"ID":"4c1bb145-37fa-4398-b3ce-507369fb22cf","Type":"ContainerStarted","Data":"40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6"}
Sep 05 07:56:23.834267 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:23.834220    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-dpgb5" podStartSLOduration=2.185723737 podStartE2EDuration="3.834206411s" totalImagesPullingTime="1.648482674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:56:23.83304903 +0000 UTC m=+464.778370837" watchObservedRunningTime="2026-09-05 07:56:23.834206411 +0000 UTC m=+464.779528216"
Sep 05 07:56:32.812220 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:32.812185    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:33.267224 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.267189    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"]
Sep 05 07:56:33.267389 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.267361    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv" podUID="4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52" containerName="limitador" containerID="cri-o://64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6" gracePeriod=30
Sep 05 07:56:33.813400 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.813377    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:33.869273 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.869198    2569 generic.go:356] "Generic (PLEG): container finished" podID="4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52" containerID="64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6" exitCode=0
Sep 05 07:56:33.869273 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.869255    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv" event={"ID":"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52","Type":"ContainerDied","Data":"64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6"}
Sep 05 07:56:33.869436 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.869293    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv" event={"ID":"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52","Type":"ContainerDied","Data":"26140fa51c73e8af679a5e276f868f986403a5c3f684540217c49d660c991a68"}
Sep 05 07:56:33.869436 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.869313    2569 scope.go:117] "RemoveContainer" containerID="64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6"
Sep 05 07:56:33.869436 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.869247    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"
Sep 05 07:56:33.877409 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.877389    2569 scope.go:117] "RemoveContainer" containerID="64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6"
Sep 05 07:56:33.877711 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:56:33.877687    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6\": container with ID starting with 64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6 not found: ID does not exist" containerID="64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6"
Sep 05 07:56:33.877806 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.877716    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6"} err="failed to get container status \"64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6\": rpc error: code = NotFound desc = could not find container \"64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6\": container with ID starting with 64edc8814d580a4e719e0e649d59075cfa51e789194b3a89b278428ad3a597b6 not found: ID does not exist"
Sep 05 07:56:33.953334 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.953313    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dsnw\" (UniqueName: \"kubernetes.io/projected/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-kube-api-access-4dsnw\") pod \"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52\" (UID: \"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52\") "
Sep 05 07:56:33.953437 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.953369    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-config-file\") pod \"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52\" (UID: \"4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52\") "
Sep 05 07:56:33.953752 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.953727    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-config-file" (OuterVolumeSpecName: "config-file") pod "4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52" (UID: "4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:56:33.955407 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:33.955389    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-kube-api-access-4dsnw" (OuterVolumeSpecName: "kube-api-access-4dsnw") pod "4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52" (UID: "4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52"). InnerVolumeSpecName "kube-api-access-4dsnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:56:34.054569 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:34.054543    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dsnw\" (UniqueName: \"kubernetes.io/projected/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-kube-api-access-4dsnw\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:56:34.054569 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:34.054565    2569 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52-config-file\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:56:34.191984 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:34.191952    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"]
Sep 05 07:56:34.197056 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:34.197033    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-bmgwv"]
Sep 05 07:56:35.569998 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:35.569963    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52" path="/var/lib/kubelet/pods/4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52/volumes"
Sep 05 07:56:36.710289 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.710258    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"]
Sep 05 07:56:36.711374 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.711347    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52" containerName="limitador"
Sep 05 07:56:36.711374 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.711376    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52" containerName="limitador"
Sep 05 07:56:36.711556 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.711481    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c7e2375-3ca0-4cb5-a78a-e9d335fbcb52" containerName="limitador"
Sep 05 07:56:36.716004 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.715979    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"
Sep 05 07:56:36.718884 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.718862    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 05 07:56:36.719218 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.719180    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 07:56:36.719218 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.719199    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4sg5l\""
Sep 05 07:56:36.719218 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.719213    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 05 07:56:36.719665 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.719645    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"]
Sep 05 07:56:36.880649 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.880620    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkwbc\" (UniqueName: \"kubernetes.io/projected/93ccecc9-4561-4de6-bf12-f2e0b900e447-kube-api-access-bkwbc\") pod \"postgres-69fb4594cf-sd9fl\" (UID: \"93ccecc9-4561-4de6-bf12-f2e0b900e447\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"
Sep 05 07:56:36.880799 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.880678    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/93ccecc9-4561-4de6-bf12-f2e0b900e447-data\") pod \"postgres-69fb4594cf-sd9fl\" (UID: \"93ccecc9-4561-4de6-bf12-f2e0b900e447\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"
Sep 05 07:56:36.981863 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.981781    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkwbc\" (UniqueName: \"kubernetes.io/projected/93ccecc9-4561-4de6-bf12-f2e0b900e447-kube-api-access-bkwbc\") pod \"postgres-69fb4594cf-sd9fl\" (UID: \"93ccecc9-4561-4de6-bf12-f2e0b900e447\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"
Sep 05 07:56:36.982070 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.981992    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/93ccecc9-4561-4de6-bf12-f2e0b900e447-data\") pod \"postgres-69fb4594cf-sd9fl\" (UID: \"93ccecc9-4561-4de6-bf12-f2e0b900e447\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"
Sep 05 07:56:36.982306 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.982288    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/93ccecc9-4561-4de6-bf12-f2e0b900e447-data\") pod \"postgres-69fb4594cf-sd9fl\" (UID: \"93ccecc9-4561-4de6-bf12-f2e0b900e447\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"
Sep 05 07:56:36.990261 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:36.990239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkwbc\" (UniqueName: \"kubernetes.io/projected/93ccecc9-4561-4de6-bf12-f2e0b900e447-kube-api-access-bkwbc\") pod \"postgres-69fb4594cf-sd9fl\" (UID: \"93ccecc9-4561-4de6-bf12-f2e0b900e447\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"
Sep 05 07:56:37.028206 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:37.028179    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"
Sep 05 07:56:37.301484 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:37.301460    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"]
Sep 05 07:56:37.303104 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:56:37.303078    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ccecc9_4561_4de6_bf12_f2e0b900e447.slice/crio-ffa64323d64dd27d426b3ecf63635ed3bee35b237f488a41d5addbc729892668 WatchSource:0}: Error finding container ffa64323d64dd27d426b3ecf63635ed3bee35b237f488a41d5addbc729892668: Status 404 returned error can't find the container with id ffa64323d64dd27d426b3ecf63635ed3bee35b237f488a41d5addbc729892668
Sep 05 07:56:37.890763 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:37.890731    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl" event={"ID":"93ccecc9-4561-4de6-bf12-f2e0b900e447","Type":"ContainerStarted","Data":"ffa64323d64dd27d426b3ecf63635ed3bee35b237f488a41d5addbc729892668"}
Sep 05 07:56:42.086596 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:42.086571    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 05 07:56:42.921485 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:42.921451    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl" event={"ID":"93ccecc9-4561-4de6-bf12-f2e0b900e447","Type":"ContainerStarted","Data":"b0ea7c574556781ed28ec2ee9995a7db472c21f53117cf9ecc66ca213e9b3ae4"}
Sep 05 07:56:42.921662 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:42.921514    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"
Sep 05 07:56:42.940727 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:42.940672    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl" podStartSLOduration=2.162002951 podStartE2EDuration="6.940652228s" totalImagesPullingTime="4.778649277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:56:42.938569232 +0000 UTC m=+483.883891041" watchObservedRunningTime="2026-09-05 07:56:42.940652228 +0000 UTC m=+483.885974034"
Sep 05 07:56:48.956126 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:48.956095    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sd9fl"
Sep 05 07:56:52.403050 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.403019    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-dpgb5"]
Sep 05 07:56:52.403542 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.403187    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-dpgb5" podUID="4c1bb145-37fa-4398-b3ce-507369fb22cf" containerName="authorino" containerID="cri-o://40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6" gracePeriod=30
Sep 05 07:56:52.655400 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.655341    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-dpgb5"
Sep 05 07:56:52.726112 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.726080    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nx45\" (UniqueName: \"kubernetes.io/projected/4c1bb145-37fa-4398-b3ce-507369fb22cf-kube-api-access-6nx45\") pod \"4c1bb145-37fa-4398-b3ce-507369fb22cf\" (UID: \"4c1bb145-37fa-4398-b3ce-507369fb22cf\") "
Sep 05 07:56:52.728289 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.728257    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c1bb145-37fa-4398-b3ce-507369fb22cf-kube-api-access-6nx45" (OuterVolumeSpecName: "kube-api-access-6nx45") pod "4c1bb145-37fa-4398-b3ce-507369fb22cf" (UID: "4c1bb145-37fa-4398-b3ce-507369fb22cf"). InnerVolumeSpecName "kube-api-access-6nx45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:56:52.827185 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.827151    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nx45\" (UniqueName: \"kubernetes.io/projected/4c1bb145-37fa-4398-b3ce-507369fb22cf-kube-api-access-6nx45\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:56:52.976267 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.976184    2569 generic.go:356] "Generic (PLEG): container finished" podID="4c1bb145-37fa-4398-b3ce-507369fb22cf" containerID="40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6" exitCode=0
Sep 05 07:56:52.976267 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.976228    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-dpgb5"
Sep 05 07:56:52.976443 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.976267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-dpgb5" event={"ID":"4c1bb145-37fa-4398-b3ce-507369fb22cf","Type":"ContainerDied","Data":"40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6"}
Sep 05 07:56:52.976443 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.976313    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-dpgb5" event={"ID":"4c1bb145-37fa-4398-b3ce-507369fb22cf","Type":"ContainerDied","Data":"4408d943717681dd8e2adbb68a64ab4c86df5d5c85cd4e01a868bdbbdedae5ce"}
Sep 05 07:56:52.976443 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.976335    2569 scope.go:117] "RemoveContainer" containerID="40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6"
Sep 05 07:56:52.985013 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.984997    2569 scope.go:117] "RemoveContainer" containerID="40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6"
Sep 05 07:56:52.985280 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:56:52.985260    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6\": container with ID starting with 40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6 not found: ID does not exist" containerID="40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6"
Sep 05 07:56:52.985350 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.985287    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6"} err="failed to get container status \"40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6\": rpc error: code = NotFound desc = could not find container \"40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6\": container with ID starting with 40e0c250a70222f6cde987a0d8654fae2e5d21ef6ae1b9a9b9bdf8280d3057c6 not found: ID does not exist"
Sep 05 07:56:52.997763 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:52.997742    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-dpgb5"]
Sep 05 07:56:53.001080 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:53.001057    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-dpgb5"]
Sep 05 07:56:53.570828 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:53.570791    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c1bb145-37fa-4398-b3ce-507369fb22cf" path="/var/lib/kubelet/pods/4c1bb145-37fa-4398-b3ce-507369fb22cf/volumes"
Sep 05 07:56:57.898694 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.898657    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b459b954d-xcwsh"]
Sep 05 07:56:57.899217 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.899203    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c1bb145-37fa-4398-b3ce-507369fb22cf" containerName="authorino"
Sep 05 07:56:57.899263 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.899220    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c1bb145-37fa-4398-b3ce-507369fb22cf" containerName="authorino"
Sep 05 07:56:57.899328 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.899317    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c1bb145-37fa-4398-b3ce-507369fb22cf" containerName="authorino"
Sep 05 07:56:57.905886 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.905865    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:57.909228 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.909208    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 05 07:56:57.909330 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.909292    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 05 07:56:57.910300 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.910284    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 05 07:56:57.910395 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.910290    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9qpdw\""
Sep 05 07:56:57.917702 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.917680    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b459b954d-xcwsh"]
Sep 05 07:56:57.972126 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.972092    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-webhook-cert\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:57.972289 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.972132    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6k8g\" (UniqueName: \"kubernetes.io/projected/6bbde6b7-34cc-45e8-8b42-87734207f153-kube-api-access-d6k8g\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:57.972289 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:57.972214    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-metrics-certs\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:58.072739 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.072701    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-webhook-cert\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:58.072914 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.072748    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6k8g\" (UniqueName: \"kubernetes.io/projected/6bbde6b7-34cc-45e8-8b42-87734207f153-kube-api-access-d6k8g\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:58.072914 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.072784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-metrics-certs\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:58.072914 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:56:58.072850    2569 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 05 07:56:58.072914 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:56:58.072902    2569 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 05 07:56:58.073066 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:56:58.072930    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-webhook-cert podName:6bbde6b7-34cc-45e8-8b42-87734207f153 nodeName:}" failed. No retries permitted until 2026-09-05 07:56:58.572910563 +0000 UTC m=+499.518232360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-webhook-cert") pod "maas-controller-7b459b954d-xcwsh" (UID: "6bbde6b7-34cc-45e8-8b42-87734207f153") : secret "maas-controller-webhook-cert" not found
Sep 05 07:56:58.073066 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:56:58.072947    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-metrics-certs podName:6bbde6b7-34cc-45e8-8b42-87734207f153 nodeName:}" failed. No retries permitted until 2026-09-05 07:56:58.57294105 +0000 UTC m=+499.518262832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-metrics-certs") pod "maas-controller-7b459b954d-xcwsh" (UID: "6bbde6b7-34cc-45e8-8b42-87734207f153") : secret "maas-controller-metrics-tls" not found
Sep 05 07:56:58.083476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.083452    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6k8g\" (UniqueName: \"kubernetes.io/projected/6bbde6b7-34cc-45e8-8b42-87734207f153-kube-api-access-d6k8g\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:58.084879 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.084855    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-jbvd9"]
Sep 05 07:56:58.093993 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.093967    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.098332 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.098301    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-jbvd9"]
Sep 05 07:56:58.173700 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.173623    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jfp5\" (UniqueName: \"kubernetes.io/projected/d730375f-48bd-48ef-b0b3-533cf1fdfba7-kube-api-access-5jfp5\") pod \"maas-controller-7cb86cc4bd-jbvd9\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") " pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.173700 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.173671    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-webhook-cert\") pod \"maas-controller-7cb86cc4bd-jbvd9\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") " pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.173885 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.173734    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-metrics-certs\") pod \"maas-controller-7cb86cc4bd-jbvd9\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") " pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.275006 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.274967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jfp5\" (UniqueName: \"kubernetes.io/projected/d730375f-48bd-48ef-b0b3-533cf1fdfba7-kube-api-access-5jfp5\") pod \"maas-controller-7cb86cc4bd-jbvd9\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") " pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.275170 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.275024    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-webhook-cert\") pod \"maas-controller-7cb86cc4bd-jbvd9\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") " pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.275170 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.275086    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-metrics-certs\") pod \"maas-controller-7cb86cc4bd-jbvd9\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") " pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.277566 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.277517    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-metrics-certs\") pod \"maas-controller-7cb86cc4bd-jbvd9\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") " pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.277676 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.277589    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-webhook-cert\") pod \"maas-controller-7cb86cc4bd-jbvd9\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") " pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.283452 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.283423    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jfp5\" (UniqueName: \"kubernetes.io/projected/d730375f-48bd-48ef-b0b3-533cf1fdfba7-kube-api-access-5jfp5\") pod \"maas-controller-7cb86cc4bd-jbvd9\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") " pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.406362 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.406310    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:56:58.578365 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.578319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-metrics-certs\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:58.578591 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.578437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-webhook-cert\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:58.580932 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.580903    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-metrics-certs\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:58.581014 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.580933    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-webhook-cert\") pod \"maas-controller-7b459b954d-xcwsh\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") " pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:58.681973 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.681951    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-jbvd9"]
Sep 05 07:56:58.683614 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:56:58.683590    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd730375f_48bd_48ef_b0b3_533cf1fdfba7.slice/crio-f13075ebd359607e2a73354b623b127c9af6105d9261bc31894d31475b0cc3a0 WatchSource:0}: Error finding container f13075ebd359607e2a73354b623b127c9af6105d9261bc31894d31475b0cc3a0: Status 404 returned error can't find the container with id f13075ebd359607e2a73354b623b127c9af6105d9261bc31894d31475b0cc3a0
Sep 05 07:56:58.817013 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:58.816912    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:56:59.006701 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:59.006665    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9" event={"ID":"d730375f-48bd-48ef-b0b3-533cf1fdfba7","Type":"ContainerStarted","Data":"f13075ebd359607e2a73354b623b127c9af6105d9261bc31894d31475b0cc3a0"}
Sep 05 07:56:59.089368 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:56:59.089320    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b459b954d-xcwsh"]
Sep 05 07:56:59.091859 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:56:59.091834    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bbde6b7_34cc_45e8_8b42_87734207f153.slice/crio-21fec15cd45e8f591cb5e654a2a09d5844fc86b5e5f9da53ece39c482164f637 WatchSource:0}: Error finding container 21fec15cd45e8f591cb5e654a2a09d5844fc86b5e5f9da53ece39c482164f637: Status 404 returned error can't find the container with id 21fec15cd45e8f591cb5e654a2a09d5844fc86b5e5f9da53ece39c482164f637
Sep 05 07:57:00.016982 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:00.016908    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b459b954d-xcwsh" event={"ID":"6bbde6b7-34cc-45e8-8b42-87734207f153","Type":"ContainerStarted","Data":"21fec15cd45e8f591cb5e654a2a09d5844fc86b5e5f9da53ece39c482164f637"}
Sep 05 07:57:02.029692 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:02.029654    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9" event={"ID":"d730375f-48bd-48ef-b0b3-533cf1fdfba7","Type":"ContainerStarted","Data":"ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e"}
Sep 05 07:57:02.030119 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:02.029753    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:57:02.031320 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:02.031287    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b459b954d-xcwsh" event={"ID":"6bbde6b7-34cc-45e8-8b42-87734207f153","Type":"ContainerStarted","Data":"9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0"}
Sep 05 07:57:02.031436 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:02.031393    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:57:02.054265 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:02.054222    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9" podStartSLOduration=1.330542997 podStartE2EDuration="4.054209467s" totalImagesPullingTime="2.72366647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:57:02.052618744 +0000 UTC m=+502.997940550" watchObservedRunningTime="2026-09-05 07:57:02.054209467 +0000 UTC m=+502.999531271"
Sep 05 07:57:02.073987 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:02.073942    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b459b954d-xcwsh" podStartSLOduration=2.751802571 podStartE2EDuration="5.073926733s" totalImagesPullingTime="2.322124162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:57:02.072233137 +0000 UTC m=+503.017554943" watchObservedRunningTime="2026-09-05 07:57:02.073926733 +0000 UTC m=+503.019248538"
Sep 05 07:57:13.043040 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.043006    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:57:13.043478 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.043083    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 07:57:13.111939 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.111903    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b459b954d-xcwsh"]
Sep 05 07:57:13.112184 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.112152    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b459b954d-xcwsh" podUID="6bbde6b7-34cc-45e8-8b42-87734207f153" containerName="manager" containerID="cri-o://9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0" gracePeriod=30
Sep 05 07:57:13.367777 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.367753    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:57:13.520836 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.520806    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-metrics-certs\") pod \"6bbde6b7-34cc-45e8-8b42-87734207f153\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") "
Sep 05 07:57:13.521026 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.520894    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6k8g\" (UniqueName: \"kubernetes.io/projected/6bbde6b7-34cc-45e8-8b42-87734207f153-kube-api-access-d6k8g\") pod \"6bbde6b7-34cc-45e8-8b42-87734207f153\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") "
Sep 05 07:57:13.521026 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.520920    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-webhook-cert\") pod \"6bbde6b7-34cc-45e8-8b42-87734207f153\" (UID: \"6bbde6b7-34cc-45e8-8b42-87734207f153\") "
Sep 05 07:57:13.523222 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.523185    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bbde6b7-34cc-45e8-8b42-87734207f153-kube-api-access-d6k8g" (OuterVolumeSpecName: "kube-api-access-d6k8g") pod "6bbde6b7-34cc-45e8-8b42-87734207f153" (UID: "6bbde6b7-34cc-45e8-8b42-87734207f153"). InnerVolumeSpecName "kube-api-access-d6k8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:57:13.523329 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.523216    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6bbde6b7-34cc-45e8-8b42-87734207f153" (UID: "6bbde6b7-34cc-45e8-8b42-87734207f153"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:57:13.523329 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.523233    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6bbde6b7-34cc-45e8-8b42-87734207f153" (UID: "6bbde6b7-34cc-45e8-8b42-87734207f153"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:57:13.622391 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.622314    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6k8g\" (UniqueName: \"kubernetes.io/projected/6bbde6b7-34cc-45e8-8b42-87734207f153-kube-api-access-d6k8g\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:57:13.622391 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.622342    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-webhook-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:57:13.622391 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:13.622352    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bbde6b7-34cc-45e8-8b42-87734207f153-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 07:57:14.100419 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:14.100389    2569 generic.go:356] "Generic (PLEG): container finished" podID="6bbde6b7-34cc-45e8-8b42-87734207f153" containerID="9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0" exitCode=0
Sep 05 07:57:14.100829 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:14.100436    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b459b954d-xcwsh"
Sep 05 07:57:14.100829 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:14.100478    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b459b954d-xcwsh" event={"ID":"6bbde6b7-34cc-45e8-8b42-87734207f153","Type":"ContainerDied","Data":"9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0"}
Sep 05 07:57:14.100829 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:14.100551    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b459b954d-xcwsh" event={"ID":"6bbde6b7-34cc-45e8-8b42-87734207f153","Type":"ContainerDied","Data":"21fec15cd45e8f591cb5e654a2a09d5844fc86b5e5f9da53ece39c482164f637"}
Sep 05 07:57:14.100829 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:14.100577    2569 scope.go:117] "RemoveContainer" containerID="9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0"
Sep 05 07:57:14.109540 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:14.109507    2569 scope.go:117] "RemoveContainer" containerID="9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0"
Sep 05 07:57:14.109829 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:57:14.109807    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0\": container with ID starting with 9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0 not found: ID does not exist" containerID="9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0"
Sep 05 07:57:14.109907 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:14.109834    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0"} err="failed to get container status \"9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0\": rpc error: code = NotFound desc = could not find container \"9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0\": container with ID starting with 9ade3f7f2e3b3ce91111ef03f050352cff2818ac82782c5649d789cc794d6ac0 not found: ID does not exist"
Sep 05 07:57:14.117243 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:14.117221    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b459b954d-xcwsh"]
Sep 05 07:57:14.121384 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:14.121363    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b459b954d-xcwsh"]
Sep 05 07:57:15.569895 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:15.569863    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bbde6b7-34cc-45e8-8b42-87734207f153" path="/var/lib/kubelet/pods/6bbde6b7-34cc-45e8-8b42-87734207f153/volumes"
Sep 05 07:57:35.571340 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.571262    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7548c976dd-msncz"]
Sep 05 07:57:35.571826 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.571644    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bbde6b7-34cc-45e8-8b42-87734207f153" containerName="manager"
Sep 05 07:57:35.571826 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.571656    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bbde6b7-34cc-45e8-8b42-87734207f153" containerName="manager"
Sep 05 07:57:35.571826 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.571727    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bbde6b7-34cc-45e8-8b42-87734207f153" containerName="manager"
Sep 05 07:57:35.578215 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.578193    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.581027 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.581005    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 05 07:57:35.581141 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.581005    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 05 07:57:35.585380 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.585357    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7548c976dd-msncz"]
Sep 05 07:57:35.619085 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.619055    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-maas-api-tls\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.619245 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.619090    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2gcv\" (UniqueName: \"kubernetes.io/projected/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-kube-api-access-x2gcv\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.619245 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.619109    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-metrics-certs\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.619245 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.619136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-openshift-service-ca\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.720272 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.720241    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-maas-api-tls\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.720458 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.720278    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2gcv\" (UniqueName: \"kubernetes.io/projected/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-kube-api-access-x2gcv\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.720458 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.720299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-metrics-certs\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.720458 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.720320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-openshift-service-ca\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.720458 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:57:35.720429    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 05 07:57:35.720796 ip-10-0-140-64 kubenswrapper[2569]: E0905 07:57:35.720518    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-metrics-certs podName:40c5a8b8-1aa9-4c59-b539-1d7db63d3515 nodeName:}" failed. No retries permitted until 2026-09-05 07:57:36.22049592 +0000 UTC m=+537.165817710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-metrics-certs") pod "maas-api-7548c976dd-msncz" (UID: "40c5a8b8-1aa9-4c59-b539-1d7db63d3515") : secret "maas-api-metrics-tls" not found
Sep 05 07:57:35.721076 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.721058    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-openshift-service-ca\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.722992 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.722972    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-maas-api-tls\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:35.730633 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:35.730608    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2gcv\" (UniqueName: \"kubernetes.io/projected/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-kube-api-access-x2gcv\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:36.101045 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.101009    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"]
Sep 05 07:57:36.104719 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.104702    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"
Sep 05 07:57:36.108292 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.108272    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7d4h6\""
Sep 05 07:57:36.108419 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.108404    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 05 07:57:36.115905 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.115880    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"]
Sep 05 07:57:36.127714 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.127684    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"]
Sep 05 07:57:36.138023 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.137978    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"
Sep 05 07:57:36.140378 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.140355    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"]
Sep 05 07:57:36.225901 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.225863    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nrbh\" (UniqueName: \"kubernetes.io/projected/31f840d8-c593-439e-9144-d2004574c575-kube-api-access-2nrbh\") pod \"payload-processing-5b54b7fbb7-fxvvn\" (UID: \"31f840d8-c593-439e-9144-d2004574c575\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"
Sep 05 07:57:36.226129 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.226009    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2chw\" (UniqueName: \"kubernetes.io/projected/34d3e8dc-f864-499f-bd75-097835e087ad-kube-api-access-w2chw\") pod \"payload-pre-processing-59fbc47bf8-lvpvq\" (UID: \"34d3e8dc-f864-499f-bd75-097835e087ad\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"
Sep 05 07:57:36.226129 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.226078    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-metrics-certs\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:36.226415 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.226136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34d3e8dc-f864-499f-bd75-097835e087ad-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-lvpvq\" (UID: \"34d3e8dc-f864-499f-bd75-097835e087ad\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"
Sep 05 07:57:36.226415 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.226212    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f840d8-c593-439e-9144-d2004574c575-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-fxvvn\" (UID: \"31f840d8-c593-439e-9144-d2004574c575\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"
Sep 05 07:57:36.228904 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.228882    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-metrics-certs\") pod \"maas-api-7548c976dd-msncz\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") " pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:36.327296 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.327262    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2chw\" (UniqueName: \"kubernetes.io/projected/34d3e8dc-f864-499f-bd75-097835e087ad-kube-api-access-w2chw\") pod \"payload-pre-processing-59fbc47bf8-lvpvq\" (UID: \"34d3e8dc-f864-499f-bd75-097835e087ad\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"
Sep 05 07:57:36.327476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.327320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34d3e8dc-f864-499f-bd75-097835e087ad-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-lvpvq\" (UID: \"34d3e8dc-f864-499f-bd75-097835e087ad\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"
Sep 05 07:57:36.327476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.327377    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f840d8-c593-439e-9144-d2004574c575-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-fxvvn\" (UID: \"31f840d8-c593-439e-9144-d2004574c575\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"
Sep 05 07:57:36.327476 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.327409    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nrbh\" (UniqueName: \"kubernetes.io/projected/31f840d8-c593-439e-9144-d2004574c575-kube-api-access-2nrbh\") pod \"payload-processing-5b54b7fbb7-fxvvn\" (UID: \"31f840d8-c593-439e-9144-d2004574c575\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"
Sep 05 07:57:36.328105 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.328080    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f840d8-c593-439e-9144-d2004574c575-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-fxvvn\" (UID: \"31f840d8-c593-439e-9144-d2004574c575\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"
Sep 05 07:57:36.328279 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.328257    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34d3e8dc-f864-499f-bd75-097835e087ad-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-lvpvq\" (UID: \"34d3e8dc-f864-499f-bd75-097835e087ad\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"
Sep 05 07:57:36.337189 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.337168    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nrbh\" (UniqueName: \"kubernetes.io/projected/31f840d8-c593-439e-9144-d2004574c575-kube-api-access-2nrbh\") pod \"payload-processing-5b54b7fbb7-fxvvn\" (UID: \"31f840d8-c593-439e-9144-d2004574c575\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"
Sep 05 07:57:36.337371 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.337253    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2chw\" (UniqueName: \"kubernetes.io/projected/34d3e8dc-f864-499f-bd75-097835e087ad-kube-api-access-w2chw\") pod \"payload-pre-processing-59fbc47bf8-lvpvq\" (UID: \"34d3e8dc-f864-499f-bd75-097835e087ad\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"
Sep 05 07:57:36.416981 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.416900    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"
Sep 05 07:57:36.449057 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.449028    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"
Sep 05 07:57:36.491243 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.491147    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:36.695547 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.695497    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"]
Sep 05 07:57:36.697675 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:57:36.697646    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d3e8dc_f864_499f_bd75_097835e087ad.slice/crio-3b8d884acc763eb2df8d8325ec49120023d2286a9fb37d9f7c4a14e0adba88f8 WatchSource:0}: Error finding container 3b8d884acc763eb2df8d8325ec49120023d2286a9fb37d9f7c4a14e0adba88f8: Status 404 returned error can't find the container with id 3b8d884acc763eb2df8d8325ec49120023d2286a9fb37d9f7c4a14e0adba88f8
Sep 05 07:57:36.736283 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.736235    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"]
Sep 05 07:57:36.739203 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:57:36.739175    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f840d8_c593_439e_9144_d2004574c575.slice/crio-44d5093777d45c093589ddb90686576c9d7a24dfa6c9cc8f0ae2a3651dfe4b6b WatchSource:0}: Error finding container 44d5093777d45c093589ddb90686576c9d7a24dfa6c9cc8f0ae2a3651dfe4b6b: Status 404 returned error can't find the container with id 44d5093777d45c093589ddb90686576c9d7a24dfa6c9cc8f0ae2a3651dfe4b6b
Sep 05 07:57:36.775021 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:36.774998    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7548c976dd-msncz"]
Sep 05 07:57:36.776760 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:57:36.776737    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c5a8b8_1aa9_4c59_b539_1d7db63d3515.slice/crio-23d4fc67bc3a002df04dfceded7077a697fda6e09d2f1daccd308d0157420f0d WatchSource:0}: Error finding container 23d4fc67bc3a002df04dfceded7077a697fda6e09d2f1daccd308d0157420f0d: Status 404 returned error can't find the container with id 23d4fc67bc3a002df04dfceded7077a697fda6e09d2f1daccd308d0157420f0d
Sep 05 07:57:37.219864 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:37.219775    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz" event={"ID":"40c5a8b8-1aa9-4c59-b539-1d7db63d3515","Type":"ContainerStarted","Data":"23d4fc67bc3a002df04dfceded7077a697fda6e09d2f1daccd308d0157420f0d"}
Sep 05 07:57:37.223409 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:37.223333    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn" event={"ID":"31f840d8-c593-439e-9144-d2004574c575","Type":"ContainerStarted","Data":"44d5093777d45c093589ddb90686576c9d7a24dfa6c9cc8f0ae2a3651dfe4b6b"}
Sep 05 07:57:37.225563 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:37.225486    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq" event={"ID":"34d3e8dc-f864-499f-bd75-097835e087ad","Type":"ContainerStarted","Data":"3b8d884acc763eb2df8d8325ec49120023d2286a9fb37d9f7c4a14e0adba88f8"}
Sep 05 07:57:40.269641 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:40.269595    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq" event={"ID":"34d3e8dc-f864-499f-bd75-097835e087ad","Type":"ContainerStarted","Data":"717227062bdb79f06b6923b785b39fa5e80fe0f834719e988cc9025897d5f54f"}
Sep 05 07:57:40.270434 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:40.270409    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"
Sep 05 07:57:40.271960 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:40.271930    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz" event={"ID":"40c5a8b8-1aa9-4c59-b539-1d7db63d3515","Type":"ContainerStarted","Data":"1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c"}
Sep 05 07:57:40.272103 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:40.271995    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:40.273386 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:40.273365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn" event={"ID":"31f840d8-c593-439e-9144-d2004574c575","Type":"ContainerStarted","Data":"19734c718eb8b0e23c02e63dc2b7816be213363653c16a8cf28f101f251b1c6c"}
Sep 05 07:57:40.273488 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:40.273477    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"
Sep 05 07:57:40.296026 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:40.295963    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq" podStartSLOduration=1.3751120700000001 podStartE2EDuration="4.295944441s" totalImagesPullingTime="2.920832371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:57:40.287257633 +0000 UTC m=+541.232579438" watchObservedRunningTime="2026-09-05 07:57:40.295944441 +0000 UTC m=+541.241266247"
Sep 05 07:57:40.306800 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:40.306751    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn" podStartSLOduration=1.433547855 podStartE2EDuration="4.3067398s" totalImagesPullingTime="2.873191945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:57:40.305508066 +0000 UTC m=+541.250829864" watchObservedRunningTime="2026-09-05 07:57:40.3067398 +0000 UTC m=+541.252061606"
Sep 05 07:57:40.323721 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:40.323673    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz" podStartSLOduration=2.48832199 podStartE2EDuration="5.323662253s" totalImagesPullingTime="2.835340263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:57:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:57:40.323104177 +0000 UTC m=+541.268425983" watchObservedRunningTime="2026-09-05 07:57:40.323662253 +0000 UTC m=+541.268984059"
Sep 05 07:57:46.284737 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:46.284701    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 07:57:46.318164 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:46.318127    2569 generic.go:356] "Generic (PLEG): container finished" podID="2b378fb5-4133-490b-af8f-065518cc9b0a" containerID="846b2cf9554e564668c155db27be4a5cd7e1d473ea793a70505d30305964aff1" exitCode=0
Sep 05 07:57:46.318312 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:46.318183    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86" event={"ID":"2b378fb5-4133-490b-af8f-065518cc9b0a","Type":"ContainerDied","Data":"846b2cf9554e564668c155db27be4a5cd7e1d473ea793a70505d30305964aff1"}
Sep 05 07:57:46.318759 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:46.318741    2569 scope.go:117] "RemoveContainer" containerID="846b2cf9554e564668c155db27be4a5cd7e1d473ea793a70505d30305964aff1"
Sep 05 07:57:46.319884 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:46.319854    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 07:57:46.319956 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:46.319936    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 07:57:46.320009 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:46.319991    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 07:57:47.325779 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:47.325740    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86" event={"ID":"2b378fb5-4133-490b-af8f-065518cc9b0a","Type":"ContainerStarted","Data":"ad245cb33b8955142c51359d85bffb406ecdf82e974734691754001363918188"}
Sep 05 07:57:47.881615 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:47.881577    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:57:47.881789 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:47.881758    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-bzs86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 05 07:57:47.881855 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:47.881831    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86" podUID="2b378fb5-4133-490b-af8f-065518cc9b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 05 07:57:48.882130 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:48.882098    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-bzs86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 05 07:57:48.882609 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:48.882176    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86" podUID="2b378fb5-4133-490b-af8f-065518cc9b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 05 07:57:49.882287 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:49.882253    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-bzs86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 05 07:57:49.882693 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:49.882314    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86" podUID="2b378fb5-4133-490b-af8f-065518cc9b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 05 07:57:50.881680 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:50.881640    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-bzs86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 05 07:57:50.881880 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:50.881712    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86" podUID="2b378fb5-4133-490b-af8f-065518cc9b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 05 07:57:51.281780 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:51.281687    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-fxvvn"
Sep 05 07:57:51.881994 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:51.881959    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-bzs86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 05 07:57:51.882163 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:51.882012    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86" podUID="2b378fb5-4133-490b-af8f-065518cc9b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 05 07:57:52.286129 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:52.286046    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lvpvq"
Sep 05 07:57:52.885951 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:52.885923    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:57:52.886319 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:52.886294    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:57:52.886980 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:52.886962    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bzs86"
Sep 05 07:57:55.995135 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:55.995104    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"]
Sep 05 07:57:55.998389 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:55.998366    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.001412 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.001392    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 05 07:57:56.002925 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.002909    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7nfpk\""
Sep 05 07:57:56.003027 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.002909    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 05 07:57:56.003027 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.002958    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 05 07:57:56.010793 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.010775    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"]
Sep 05 07:57:56.045075 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.045052    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x9rr\" (UniqueName: \"kubernetes.io/projected/8c44b3ff-907f-422e-bf0d-b03efb882c7a-kube-api-access-4x9rr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.045206 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.045091    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.045206 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.045117    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.045335 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.045224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.045335 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.045292    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.045335 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.045324    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c44b3ff-907f-422e-bf0d-b03efb882c7a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.145943 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.145917    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.146073 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.145951    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.146073 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.145983    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.146073 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.146000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c44b3ff-907f-422e-bf0d-b03efb882c7a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.146073 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.146036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x9rr\" (UniqueName: \"kubernetes.io/projected/8c44b3ff-907f-422e-bf0d-b03efb882c7a-kube-api-access-4x9rr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.146073 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.146063    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.146373 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.146352    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.146423 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.146405    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.146551 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.146502    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.148409 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.148381    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c44b3ff-907f-422e-bf0d-b03efb882c7a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.148700 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.148680    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c44b3ff-907f-422e-bf0d-b03efb882c7a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.170209 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.170190    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x9rr\" (UniqueName: \"kubernetes.io/projected/8c44b3ff-907f-422e-bf0d-b03efb882c7a-kube-api-access-4x9rr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b\" (UID: \"8c44b3ff-907f-422e-bf0d-b03efb882c7a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.313305 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.313235    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:57:56.595505 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:56.595478    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"]
Sep 05 07:57:56.597232 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:57:56.597205    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c44b3ff_907f_422e_bf0d_b03efb882c7a.slice/crio-21cc24850703edd399c051000e971affad8cb809c1d0449fe5e7f1251061ea48 WatchSource:0}: Error finding container 21cc24850703edd399c051000e971affad8cb809c1d0449fe5e7f1251061ea48: Status 404 returned error can't find the container with id 21cc24850703edd399c051000e971affad8cb809c1d0449fe5e7f1251061ea48
Sep 05 07:57:57.400935 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:57.400891    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b" event={"ID":"8c44b3ff-907f-422e-bf0d-b03efb882c7a","Type":"ContainerStarted","Data":"21cc24850703edd399c051000e971affad8cb809c1d0449fe5e7f1251061ea48"}
Sep 05 07:57:58.408093 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:58.408063    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b" event={"ID":"8c44b3ff-907f-422e-bf0d-b03efb882c7a","Type":"ContainerStarted","Data":"69323612f20e84c0cc4b2b3a2b4d084c4f92676e650a304168f6748e3a467afc"}
Sep 05 07:57:58.427115 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:57:58.427055    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b" podStartSLOduration=2.490440823 podStartE2EDuration="3.427040433s" totalImagesPullingTime="936.59961ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:57:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:57:58.424460733 +0000 UTC m=+559.369782538" watchObservedRunningTime="2026-09-05 07:57:58.427040433 +0000 UTC m=+559.372362237"
Sep 05 07:58:06.313823 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:06.313787    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:58:06.314228 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:06.313839    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:58:06.326269 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:06.326245    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:58:06.460248 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:06.460225    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b"
Sep 05 07:58:21.879592 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:21.879555    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"]
Sep 05 07:58:21.885904 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:21.885880    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:21.889409 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:21.889374    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 05 07:58:21.891400 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:21.891376    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"]
Sep 05 07:58:21.974347 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:21.974324    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:21.974508 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:21.974354    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q64p4\" (UniqueName: \"kubernetes.io/projected/4f70be3c-1197-491b-9e67-acb8468f288f-kube-api-access-q64p4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:21.974508 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:21.974409    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:21.974508 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:21.974438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:21.974508 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:21.974460    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:21.974707 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:21.974552    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f70be3c-1197-491b-9e67-acb8468f288f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.075139 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.075104    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.075290 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.075150    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.075290 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.075175    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.075290 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.075196    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f70be3c-1197-491b-9e67-acb8468f288f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.075290 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.075223    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.075290 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.075240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q64p4\" (UniqueName: \"kubernetes.io/projected/4f70be3c-1197-491b-9e67-acb8468f288f-kube-api-access-q64p4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.075669 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.075649    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.075734 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.075715    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.075853 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.075767    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.077741 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.077712    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f70be3c-1197-491b-9e67-acb8468f288f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.077990 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.077974    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f70be3c-1197-491b-9e67-acb8468f288f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.087167 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.087145    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q64p4\" (UniqueName: \"kubernetes.io/projected/4f70be3c-1197-491b-9e67-acb8468f288f-kube-api-access-q64p4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nd85f\" (UID: \"4f70be3c-1197-491b-9e67-acb8468f288f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.198969 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.198918    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:22.470969 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.470945    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"]
Sep 05 07:58:22.472799 ip-10-0-140-64 kubenswrapper[2569]: W0905 07:58:22.472765    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f70be3c_1197_491b_9e67_acb8468f288f.slice/crio-8908374d018a19c3d2a96733eae215aebaacf45aef7aafba5915766faf573626 WatchSource:0}: Error finding container 8908374d018a19c3d2a96733eae215aebaacf45aef7aafba5915766faf573626: Status 404 returned error can't find the container with id 8908374d018a19c3d2a96733eae215aebaacf45aef7aafba5915766faf573626
Sep 05 07:58:22.537634 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:22.537600    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f" event={"ID":"4f70be3c-1197-491b-9e67-acb8468f288f","Type":"ContainerStarted","Data":"8908374d018a19c3d2a96733eae215aebaacf45aef7aafba5915766faf573626"}
Sep 05 07:58:23.544884 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:23.544848    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f" event={"ID":"4f70be3c-1197-491b-9e67-acb8468f288f","Type":"ContainerStarted","Data":"60275b6dcf54660741e8071e752d83d342979ed217a92c44b38cc5549f980e5d"}
Sep 05 07:58:23.568135 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:23.568075    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f" podStartSLOduration=2.405090335 podStartE2EDuration="2.568059697s" totalImagesPullingTime="162.969362ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:58:23.567569025 +0000 UTC m=+584.512890826" watchObservedRunningTime="2026-09-05 07:58:23.568059697 +0000 UTC m=+584.513381502"
Sep 05 07:58:32.200028 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:32.199993    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:32.200589 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:32.200069    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:32.212410 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:32.212389    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:32.607090 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:32.607061    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nd85f"
Sep 05 07:58:39.532022 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:39.531993    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 07:58:39.532611 ip-10-0-140-64 kubenswrapper[2569]: I0905 07:58:39.532329    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:00:39.561811 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.561734    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"]
Sep 05 08:00:39.567873 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.567422    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.571498 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.571476    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e7c0ff-openshift-default-dockercfg-m44zt\""
Sep 05 08:00:39.587610 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.587581    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"]
Sep 05 08:00:39.706339 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.706302    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94qld\" (UniqueName: \"kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-kube-api-access-94qld\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.706512 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.706382    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-data\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.706512 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.706402    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-certs\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.706512 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.706421    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-envoy\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.706512 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.706438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istiod-ca-cert\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.706692 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.706558    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-credential-socket\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.706692 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.706605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-podinfo\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.706692 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.706635    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-token\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.706692 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.706658    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-socket\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.807707 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.807667    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istiod-ca-cert\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.807904 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.807730    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-credential-socket\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.807904 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.807757    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-podinfo\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.807904 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.807775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-token\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.807904 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.807804    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-socket\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.807904 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.807861    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94qld\" (UniqueName: \"kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-kube-api-access-94qld\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.808214 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.807918    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-data\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.808214 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.807947    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-certs\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.808214 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.807990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-envoy\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.808390 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.808261    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-credential-socket\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.808562 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.808520    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istiod-ca-cert\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.808687 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.808631    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-certs\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.808758 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.808557    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-socket\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.808758 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.808728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-data\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.810338 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.810315    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-envoy\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.810423 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.810367    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-podinfo\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.817369 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.817309    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-token\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.817866 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.817844    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94qld\" (UniqueName: \"kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-kube-api-access-94qld\") pod \"e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") " pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:39.878252 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:39.878220    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:40.168657 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:40.168631    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"]
Sep 05 08:00:40.170815 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:00:40.170787    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9e4a1d_13c5_4bef_ab2d_ab6775fa0304.slice/crio-203f120a5d503fe4afe3a0f2388cf25dec14b2b7e3a0f96dc0ddd89cb2defe93 WatchSource:0}: Error finding container 203f120a5d503fe4afe3a0f2388cf25dec14b2b7e3a0f96dc0ddd89cb2defe93: Status 404 returned error can't find the container with id 203f120a5d503fe4afe3a0f2388cf25dec14b2b7e3a0f96dc0ddd89cb2defe93
Sep 05 08:00:40.173021 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:40.172990    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:00:40.173102 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:40.173053    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:00:40.173102 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:40.173079    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:00:40.256510 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:40.256480    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" event={"ID":"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304","Type":"ContainerStarted","Data":"a1a01ad20ba71d7da197d66b1399f7249e2fbbe1a4e5b0a13e4284b6e581f1eb"}
Sep 05 08:00:40.256676 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:40.256556    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" event={"ID":"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304","Type":"ContainerStarted","Data":"203f120a5d503fe4afe3a0f2388cf25dec14b2b7e3a0f96dc0ddd89cb2defe93"}
Sep 05 08:00:40.279221 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:40.279158    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" podStartSLOduration=1.27913451 podStartE2EDuration="1.27913451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:00:40.278672803 +0000 UTC m=+721.223994609" watchObservedRunningTime="2026-09-05 08:00:40.27913451 +0000 UTC m=+721.224456317"
Sep 05 08:00:40.878617 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:40.878583    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:41.883830 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:41.883797    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:42.267861 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:42.267765    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:42.268661 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:42.268637    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:00:47.922358 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:47.922325    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"]
Sep 05 08:00:47.926476 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:47.926459    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:47.930179 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:47.930145    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-73e112cd-openshift-default-dockercfg-bw6xc\""
Sep 05 08:00:47.941889 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:47.941859    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"]
Sep 05 08:00:48.096536 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.096489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-podinfo\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.096709 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.096553    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-data\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.096709 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.096623    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-certs\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.096709 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.096682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwk2d\" (UniqueName: \"kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-kube-api-access-hwk2d\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.096829 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.096728    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-socket\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.096829 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.096763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-token\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.096829 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.096785    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-credential-socket\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.096829 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.096815    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istiod-ca-cert\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.096969 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.096830    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-envoy\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.197916 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.197822    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istiod-ca-cert\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.197916 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.197857    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-envoy\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198137 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.197992    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-podinfo\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198137 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-data\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198137 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-certs\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198307 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198174    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwk2d\" (UniqueName: \"kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-kube-api-access-hwk2d\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198307 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198217    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-socket\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198307 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198266    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-token\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198307 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-credential-socket\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198584 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198396    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-data\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198657 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198606    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-certs\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198767 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198744    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istiod-ca-cert\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198839 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198807    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-credential-socket\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.198889 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.198847    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-socket\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.200374 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.200345    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-envoy\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.200896 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.200877    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-podinfo\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.207861 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.207843    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwk2d\" (UniqueName: \"kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-kube-api-access-hwk2d\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.207951 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.207940    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-token\") pod \"e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") " pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.238145 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.238115    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:48.518066 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.517997    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"]
Sep 05 08:00:48.520513 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:00:48.520490    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1eb2a5a_8f15_419b_859d_472fdf10a8f8.slice/crio-da43b3c668379bc7475c746e5af09256fb6fc492e02d819bc91db67ec5c73395 WatchSource:0}: Error finding container da43b3c668379bc7475c746e5af09256fb6fc492e02d819bc91db67ec5c73395: Status 404 returned error can't find the container with id da43b3c668379bc7475c746e5af09256fb6fc492e02d819bc91db67ec5c73395
Sep 05 08:00:48.522854 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.522821    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:00:48.522965 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.522886    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:00:48.522965 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:48.522915    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:00:49.307765 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:49.307735    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" event={"ID":"b1eb2a5a-8f15-419b-859d-472fdf10a8f8","Type":"ContainerStarted","Data":"ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6"}
Sep 05 08:00:49.308137 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:49.307785    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" event={"ID":"b1eb2a5a-8f15-419b-859d-472fdf10a8f8","Type":"ContainerStarted","Data":"da43b3c668379bc7475c746e5af09256fb6fc492e02d819bc91db67ec5c73395"}
Sep 05 08:00:49.331325 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:49.331284    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" podStartSLOduration=2.331268669 podStartE2EDuration="2.331268669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:00:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:00:49.32930787 +0000 UTC m=+730.274629688" watchObservedRunningTime="2026-09-05 08:00:49.331268669 +0000 UTC m=+730.276590473"
Sep 05 08:00:50.238745 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:50.238708    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:50.243819 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:50.243795    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:50.317070 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:50.317038    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:50.318054 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:50.318036    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:00:51.809826 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.809794    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"]
Sep 05 08:00:51.813829 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.813808    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.816964 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.816943    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c62fz\""
Sep 05 08:00:51.817090 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.816962    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-73e112cd-serving-cert\""
Sep 05 08:00:51.824843 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.824818    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"]
Sep 05 08:00:51.840590 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.840560    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-maas-api-tls\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.841970 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.841948    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-metrics-certs\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.843002 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.842968    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/795ef037-692e-44eb-9040-d6e598663c7a-openshift-service-ca\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.843328 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.843310    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw6v9\" (UniqueName: \"kubernetes.io/projected/795ef037-692e-44eb-9040-d6e598663c7a-kube-api-access-qw6v9\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.947122 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.947082    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-maas-api-tls\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.947447 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.947427    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-metrics-certs\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.947693 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.947669    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/795ef037-692e-44eb-9040-d6e598663c7a-openshift-service-ca\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.947851 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.947829    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw6v9\" (UniqueName: \"kubernetes.io/projected/795ef037-692e-44eb-9040-d6e598663c7a-kube-api-access-qw6v9\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.949338 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.949316    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/795ef037-692e-44eb-9040-d6e598663c7a-openshift-service-ca\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.953547 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.951903    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-metrics-certs\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.953547 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.953273    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-maas-api-tls\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:51.966257 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:51.966198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw6v9\" (UniqueName: \"kubernetes.io/projected/795ef037-692e-44eb-9040-d6e598663c7a-kube-api-access-qw6v9\") pod \"maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:52.128069 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:52.128030    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:52.427732 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:52.427703    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"]
Sep 05 08:00:52.429758 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:00:52.429728    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795ef037_692e_44eb_9040_d6e598663c7a.slice/crio-72a4c8ae262625c18ab35d5d9622dc3f68c798384b3a28555886feaeeffafc7a WatchSource:0}: Error finding container 72a4c8ae262625c18ab35d5d9622dc3f68c798384b3a28555886feaeeffafc7a: Status 404 returned error can't find the container with id 72a4c8ae262625c18ab35d5d9622dc3f68c798384b3a28555886feaeeffafc7a
Sep 05 08:00:52.431320 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:52.431302    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 08:00:53.335113 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:53.335079    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x" event={"ID":"795ef037-692e-44eb-9040-d6e598663c7a","Type":"ContainerStarted","Data":"ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983"}
Sep 05 08:00:53.335662 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:53.335135    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x" event={"ID":"795ef037-692e-44eb-9040-d6e598663c7a","Type":"ContainerStarted","Data":"72a4c8ae262625c18ab35d5d9622dc3f68c798384b3a28555886feaeeffafc7a"}
Sep 05 08:00:53.335662 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:53.335162    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:00:53.361490 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:53.358224    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x" podStartSLOduration=1.947715141 podStartE2EDuration="2.358201892s" totalImagesPullingTime="410.486751ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:00:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:00:53.357497905 +0000 UTC m=+734.302819714" watchObservedRunningTime="2026-09-05 08:00:53.358201892 +0000 UTC m=+734.303523701"
Sep 05 08:00:55.405854 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.405824    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"]
Sep 05 08:00:55.409632 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.409613    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.413673 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.413402    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-4faf97-serving-cert\""
Sep 05 08:00:55.423181 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.423153    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"]
Sep 05 08:00:55.496405 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.496374    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-metrics-certs\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.496641 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.496420    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-maas-api-tls\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.496641 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.496593    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgmj7\" (UniqueName: \"kubernetes.io/projected/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-kube-api-access-sgmj7\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.496778 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.496645    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-openshift-service-ca\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.598060 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.598030    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-metrics-certs\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.598233 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.598072    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-maas-api-tls\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.598233 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.598149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgmj7\" (UniqueName: \"kubernetes.io/projected/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-kube-api-access-sgmj7\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.598233 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.598188    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-openshift-service-ca\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.598832 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.598810    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-openshift-service-ca\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.601067 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.601051    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-maas-api-tls\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.601299 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.601276    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-metrics-certs\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.607642 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.607620    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgmj7\" (UniqueName: \"kubernetes.io/projected/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-kube-api-access-sgmj7\") pod \"maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:55.723370 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:55.723284    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:56.030749 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:56.030720    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"]
Sep 05 08:00:56.032071 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:00:56.032047    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ed0ff2_2dfc_4dc0_af8d_549c299bc293.slice/crio-5a7d34a7584eae82a709bf8bb72b77a073e93443c4b111d26d955fc69c9969c5 WatchSource:0}: Error finding container 5a7d34a7584eae82a709bf8bb72b77a073e93443c4b111d26d955fc69c9969c5: Status 404 returned error can't find the container with id 5a7d34a7584eae82a709bf8bb72b77a073e93443c4b111d26d955fc69c9969c5
Sep 05 08:00:56.374986 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:56.374928    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf" event={"ID":"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293","Type":"ContainerStarted","Data":"5a7d34a7584eae82a709bf8bb72b77a073e93443c4b111d26d955fc69c9969c5"}
Sep 05 08:00:57.393636 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:57.389391    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf" event={"ID":"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293","Type":"ContainerStarted","Data":"c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99"}
Sep 05 08:00:57.393636 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:57.389827    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:00:57.414247 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:57.414195    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf" podStartSLOduration=1.917835073 podStartE2EDuration="2.41418013s" totalImagesPullingTime="496.345057ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:00:57.412597992 +0000 UTC m=+738.357919797" watchObservedRunningTime="2026-09-05 08:00:57.41418013 +0000 UTC m=+738.359501934"
Sep 05 08:00:59.362555 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:00:59.356129    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:01:02.422943 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:02.422911    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerID="a1a01ad20ba71d7da197d66b1399f7249e2fbbe1a4e5b0a13e4284b6e581f1eb" exitCode=0
Sep 05 08:01:02.423365 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:02.422949    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" event={"ID":"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304","Type":"ContainerDied","Data":"a1a01ad20ba71d7da197d66b1399f7249e2fbbe1a4e5b0a13e4284b6e581f1eb"}
Sep 05 08:01:02.423365 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:02.423304    2569 scope.go:117] "RemoveContainer" containerID="a1a01ad20ba71d7da197d66b1399f7249e2fbbe1a4e5b0a13e4284b6e581f1eb"
Sep 05 08:01:02.424307 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:02.424276    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:02.424408 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:02.424358    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:02.424474 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:02.424405    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:03.400414 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:03.400384    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:01:03.431390 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:03.431347    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" event={"ID":"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304","Type":"ContainerStarted","Data":"1a4c563f63aa8a0ecd5c5b94e175133bcc7ada73ea5967d94f2717f16f52cb81"}
Sep 05 08:01:03.879060 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:03.879028    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:01:03.879274 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:03.879254    2569 patch_prober.go:28] interesting pod/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 05 08:01:03.879337 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:03.879316    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 05 08:01:04.879665 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:04.879629    2569 patch_prober.go:28] interesting pod/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 05 08:01:04.880061 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:04.879683    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 05 08:01:05.879359 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:05.879316    2569 patch_prober.go:28] interesting pod/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 05 08:01:05.879519 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:05.879389    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 05 08:01:06.359072 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:06.358718    2569 patch_prober.go:28] interesting pod/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 08:01:06.359072 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:06.358798    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 08:01:06.879346 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:06.879306    2569 patch_prober.go:28] interesting pod/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 05 08:01:06.879570 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:06.879380    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 05 08:01:07.878897 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:07.878856    2569 patch_prober.go:28] interesting pod/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 05 08:01:07.879357 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:07.878915    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 05 08:01:08.475667 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:08.475634    2569 generic.go:356] "Generic (PLEG): container finished" podID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerID="ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6" exitCode=0
Sep 05 08:01:08.475904 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:08.475712    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" event={"ID":"b1eb2a5a-8f15-419b-859d-472fdf10a8f8","Type":"ContainerDied","Data":"ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6"}
Sep 05 08:01:08.476153 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:08.476136    2569 scope.go:117] "RemoveContainer" containerID="ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6"
Sep 05 08:01:08.477267 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:08.477242    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:08.477329 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:08.477317    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:08.477390 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:08.477377    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:09.073630 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:09.073593    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:01:09.074236 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:09.074211    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:01:09.075139 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:09.075115    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:01:09.483325 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:09.483236    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" event={"ID":"b1eb2a5a-8f15-419b-859d-472fdf10a8f8","Type":"ContainerStarted","Data":"565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e"}
Sep 05 08:01:10.241720 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:10.241680    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:01:10.242370 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:10.242344    2569 patch_prober.go:28] interesting pod/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 05 08:01:10.242586 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:10.242558    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 05 08:01:11.239437 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:11.239397    2569 patch_prober.go:28] interesting pod/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 05 08:01:11.239760 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:11.239725    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 05 08:01:12.238687 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:12.238652    2569 patch_prober.go:28] interesting pod/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 05 08:01:12.239165 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:12.238727    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 05 08:01:13.245957 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:13.245910    2569 patch_prober.go:28] interesting pod/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 05 08:01:13.246441 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:13.245982    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 05 08:01:14.245556 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:14.243910    2569 patch_prober.go:28] interesting pod/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 05 08:01:14.245556 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:14.243968    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 05 08:01:15.246231 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:15.246189    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:01:15.246724 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:15.246696    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:01:15.247940 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:15.247914    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:01:32.783065 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:32.783031    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"]
Sep 05 08:01:32.783648 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:32.783308    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x" podUID="795ef037-692e-44eb-9040-d6e598663c7a" containerName="maas-api" containerID="cri-o://ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983" gracePeriod=30
Sep 05 08:01:33.050585 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.050490    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:01:33.151692 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.151657    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-maas-api-tls\") pod \"795ef037-692e-44eb-9040-d6e598663c7a\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") "
Sep 05 08:01:33.151857 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.151753    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-metrics-certs\") pod \"795ef037-692e-44eb-9040-d6e598663c7a\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") "
Sep 05 08:01:33.151857 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.151821    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw6v9\" (UniqueName: \"kubernetes.io/projected/795ef037-692e-44eb-9040-d6e598663c7a-kube-api-access-qw6v9\") pod \"795ef037-692e-44eb-9040-d6e598663c7a\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") "
Sep 05 08:01:33.151953 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.151866    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/795ef037-692e-44eb-9040-d6e598663c7a-openshift-service-ca\") pod \"795ef037-692e-44eb-9040-d6e598663c7a\" (UID: \"795ef037-692e-44eb-9040-d6e598663c7a\") "
Sep 05 08:01:33.152321 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.152288    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/795ef037-692e-44eb-9040-d6e598663c7a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "795ef037-692e-44eb-9040-d6e598663c7a" (UID: "795ef037-692e-44eb-9040-d6e598663c7a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:01:33.154247 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.154216    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "795ef037-692e-44eb-9040-d6e598663c7a" (UID: "795ef037-692e-44eb-9040-d6e598663c7a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:01:33.154247 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.154222    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/795ef037-692e-44eb-9040-d6e598663c7a-kube-api-access-qw6v9" (OuterVolumeSpecName: "kube-api-access-qw6v9") pod "795ef037-692e-44eb-9040-d6e598663c7a" (UID: "795ef037-692e-44eb-9040-d6e598663c7a"). InnerVolumeSpecName "kube-api-access-qw6v9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:01:33.154370 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.154256    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "795ef037-692e-44eb-9040-d6e598663c7a" (UID: "795ef037-692e-44eb-9040-d6e598663c7a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:01:33.253410 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.253379    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/795ef037-692e-44eb-9040-d6e598663c7a-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:33.253410 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.253408    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:33.253410 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.253417    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795ef037-692e-44eb-9040-d6e598663c7a-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:33.253653 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.253424    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw6v9\" (UniqueName: \"kubernetes.io/projected/795ef037-692e-44eb-9040-d6e598663c7a-kube-api-access-qw6v9\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:33.266072 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.266046    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"]
Sep 05 08:01:33.266286 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.266242    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy" containerID="cri-o://565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e" gracePeriod=30
Sep 05 08:01:33.616287 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.616251    2569 generic.go:356] "Generic (PLEG): container finished" podID="795ef037-692e-44eb-9040-d6e598663c7a" containerID="ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983" exitCode=0
Sep 05 08:01:33.616477 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.616302    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"
Sep 05 08:01:33.616477 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.616337    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x" event={"ID":"795ef037-692e-44eb-9040-d6e598663c7a","Type":"ContainerDied","Data":"ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983"}
Sep 05 08:01:33.616477 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.616390    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x" event={"ID":"795ef037-692e-44eb-9040-d6e598663c7a","Type":"ContainerDied","Data":"72a4c8ae262625c18ab35d5d9622dc3f68c798384b3a28555886feaeeffafc7a"}
Sep 05 08:01:33.616477 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.616413    2569 scope.go:117] "RemoveContainer" containerID="ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983"
Sep 05 08:01:33.625714 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.625697    2569 scope.go:117] "RemoveContainer" containerID="ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983"
Sep 05 08:01:33.626039 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:01:33.626005    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983\": container with ID starting with ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983 not found: ID does not exist" containerID="ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983"
Sep 05 08:01:33.626094 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.626047    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983"} err="failed to get container status \"ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983\": rpc error: code = NotFound desc = could not find container \"ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983\": container with ID starting with ec92fc2e68a054b2647ac9f888548d9996fa582165c9bfb5f4231622f5238983 not found: ID does not exist"
Sep 05 08:01:33.639787 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.639762    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"]
Sep 05 08:01:33.644280 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:33.644259    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73e112cd-7d76ddbbc7-lm76x"]
Sep 05 08:01:34.264183 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.264153    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"]
Sep 05 08:01:34.264689 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.264656    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795ef037-692e-44eb-9040-d6e598663c7a" containerName="maas-api"
Sep 05 08:01:34.264689 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.264670    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="795ef037-692e-44eb-9040-d6e598663c7a" containerName="maas-api"
Sep 05 08:01:34.264816 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.264774    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="795ef037-692e-44eb-9040-d6e598663c7a" containerName="maas-api"
Sep 05 08:01:34.269142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.269115    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.272167 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.272146    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-7dd976de-gw-openshift-default-dockercfg-q7nlm\""
Sep 05 08:01:34.282754 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.282729    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"]
Sep 05 08:01:34.366249 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.366213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-podinfo\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.366415 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.366273    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-certs\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.366415 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.366355    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/859f0b10-3b9d-4e97-954b-024b82d3b870-istiod-ca-cert\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.366551 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.366450    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-socket\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.366551 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.366482    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-credential-socket\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.366645 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.366596    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-envoy\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.366688 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.366672    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-data\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.366724 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.366703    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-token\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.366764 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.366736    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tv2r\" (UniqueName: \"kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-kube-api-access-5tv2r\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.467366 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467332    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-token\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.467366 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467368    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tv2r\" (UniqueName: \"kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-kube-api-access-5tv2r\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.467661 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467404    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-podinfo\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.467661 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467428    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-certs\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.467661 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467448    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/859f0b10-3b9d-4e97-954b-024b82d3b870-istiod-ca-cert\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.467661 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467586    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-socket\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.467661 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467623    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-credential-socket\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.467894 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-envoy\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.467894 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467776    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-data\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.467894 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467884    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-certs\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.468009 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.467957    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-socket\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.468080 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.468004    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-credential-socket\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.468168 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.468147    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-data\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.468240 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.468229    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/859f0b10-3b9d-4e97-954b-024b82d3b870-istiod-ca-cert\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.470175 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.470154    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-envoy\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.470363 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.470337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-podinfo\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.482398 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.482373    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-token\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.482574 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.482515    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tv2r\" (UniqueName: \"kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-kube-api-access-5tv2r\") pod \"e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") " pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.581006 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.580975    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:34.873066 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.873024    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"]
Sep 05 08:01:34.874507 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:01:34.874482    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod859f0b10_3b9d_4e97_954b_024b82d3b870.slice/crio-7e8781530355b75157ae57a0243c15d84c74769dd9615bf066338ed03847e1b3 WatchSource:0}: Error finding container 7e8781530355b75157ae57a0243c15d84c74769dd9615bf066338ed03847e1b3: Status 404 returned error can't find the container with id 7e8781530355b75157ae57a0243c15d84c74769dd9615bf066338ed03847e1b3
Sep 05 08:01:34.876823 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.876785    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:34.876930 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.876869    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:34.876930 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:34.876908    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:35.571447 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:35.571416    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="795ef037-692e-44eb-9040-d6e598663c7a" path="/var/lib/kubelet/pods/795ef037-692e-44eb-9040-d6e598663c7a/volumes"
Sep 05 08:01:35.630132 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:35.630101    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" event={"ID":"859f0b10-3b9d-4e97-954b-024b82d3b870","Type":"ContainerStarted","Data":"04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8"}
Sep 05 08:01:35.630301 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:35.630151    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" event={"ID":"859f0b10-3b9d-4e97-954b-024b82d3b870","Type":"ContainerStarted","Data":"7e8781530355b75157ae57a0243c15d84c74769dd9615bf066338ed03847e1b3"}
Sep 05 08:01:36.581925 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:36.581892    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:36.586802 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:36.586781    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:36.609951 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:36.609910    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" podStartSLOduration=2.609895266 podStartE2EDuration="2.609895266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:01:35.664003572 +0000 UTC m=+776.609325374" watchObservedRunningTime="2026-09-05 08:01:36.609895266 +0000 UTC m=+777.555217070"
Sep 05 08:01:36.635588 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:36.635566    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:36.636414 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:36.636398    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:37.407467 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.407429    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"]
Sep 05 08:01:37.412275 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.412253    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.415352 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.415331    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-7dd976de-serving-cert\""
Sep 05 08:01:37.423704 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.423681    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"]
Sep 05 08:01:37.500589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.500552    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.500776 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.500617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-metrics-certs\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.500776 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.500743    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-624pz\" (UniqueName: \"kubernetes.io/projected/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-kube-api-access-624pz\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.500871 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.500804    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-openshift-service-ca\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.602147 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.602115    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.602694 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.602160    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-metrics-certs\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.602694 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.602209    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-624pz\" (UniqueName: \"kubernetes.io/projected/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-kube-api-access-624pz\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.602694 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.602244    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-openshift-service-ca\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.602694 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:01:37.602324    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-serving-cert: secret "maas-api-e2e-migrate-7dd976de-serving-cert" not found
Sep 05 08:01:37.602694 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:01:37.602421    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls podName:cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0 nodeName:}" failed. No retries permitted until 2026-09-05 08:01:38.102399053 +0000 UTC m=+779.047720855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls") pod "maas-api-e2e-migrate-7dd976de-5464566d84-87zs5" (UID: "cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0") : secret "maas-api-e2e-migrate-7dd976de-serving-cert" not found
Sep 05 08:01:37.603027 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.603011    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-openshift-service-ca\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.604993 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.604970    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-metrics-certs\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:37.614067 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:37.614045    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-624pz\" (UniqueName: \"kubernetes.io/projected/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-kube-api-access-624pz\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:38.108373 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.108334    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:38.108564 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:01:38.108500    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-serving-cert: secret "maas-api-e2e-migrate-7dd976de-serving-cert" not found
Sep 05 08:01:38.108617 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:01:38.108604    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls podName:cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0 nodeName:}" failed. No retries permitted until 2026-09-05 08:01:39.108584217 +0000 UTC m=+780.053906018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls") pod "maas-api-e2e-migrate-7dd976de-5464566d84-87zs5" (UID: "cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0") : secret "maas-api-e2e-migrate-7dd976de-serving-cert" not found
Sep 05 08:01:38.527114 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.527088    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:01:38.614617 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.614576    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-podinfo\") pod \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") "
Sep 05 08:01:38.614617 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.614617    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-token\") pod \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") "
Sep 05 08:01:38.615114 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.614652    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-data\") pod \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") "
Sep 05 08:01:38.615114 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.614711    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-socket\") pod \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") "
Sep 05 08:01:38.615114 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.614745    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istiod-ca-cert\") pod \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") "
Sep 05 08:01:38.615114 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.614803    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-envoy\") pod \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") "
Sep 05 08:01:38.615114 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.614871    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-credential-socket\") pod \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") "
Sep 05 08:01:38.615114 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.614916    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwk2d\" (UniqueName: \"kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-kube-api-access-hwk2d\") pod \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") "
Sep 05 08:01:38.615114 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.614964    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-certs\") pod \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\" (UID: \"b1eb2a5a-8f15-419b-859d-472fdf10a8f8\") "
Sep 05 08:01:38.615114 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.615003    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1eb2a5a-8f15-419b-859d-472fdf10a8f8" (UID: "b1eb2a5a-8f15-419b-859d-472fdf10a8f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:01:38.615114 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.615018    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1eb2a5a-8f15-419b-859d-472fdf10a8f8" (UID: "b1eb2a5a-8f15-419b-859d-472fdf10a8f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:01:38.615590 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.615190    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1eb2a5a-8f15-419b-859d-472fdf10a8f8" (UID: "b1eb2a5a-8f15-419b-859d-472fdf10a8f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:01:38.615590 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.615211    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1eb2a5a-8f15-419b-859d-472fdf10a8f8" (UID: "b1eb2a5a-8f15-419b-859d-472fdf10a8f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:01:38.615590 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.615312    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1eb2a5a-8f15-419b-859d-472fdf10a8f8" (UID: "b1eb2a5a-8f15-419b-859d-472fdf10a8f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:01:38.615694 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.615654    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:38.615694 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.615674    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:38.615694 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.615691    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:38.615791 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.615702    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:38.615791 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.615716    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:38.617201 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.617175    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1eb2a5a-8f15-419b-859d-472fdf10a8f8" (UID: "b1eb2a5a-8f15-419b-859d-472fdf10a8f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:01:38.617339 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.617215    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-kube-api-access-hwk2d" (OuterVolumeSpecName: "kube-api-access-hwk2d") pod "b1eb2a5a-8f15-419b-859d-472fdf10a8f8" (UID: "b1eb2a5a-8f15-419b-859d-472fdf10a8f8"). InnerVolumeSpecName "kube-api-access-hwk2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:01:38.617457 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.617443    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1eb2a5a-8f15-419b-859d-472fdf10a8f8" (UID: "b1eb2a5a-8f15-419b-859d-472fdf10a8f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:01:38.617610 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.617589    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1eb2a5a-8f15-419b-859d-472fdf10a8f8" (UID: "b1eb2a5a-8f15-419b-859d-472fdf10a8f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:01:38.651664 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.651594    2569 generic.go:356] "Generic (PLEG): container finished" podID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerID="565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e" exitCode=0
Sep 05 08:01:38.651664 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.651640    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"
Sep 05 08:01:38.651835 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.651678    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" event={"ID":"b1eb2a5a-8f15-419b-859d-472fdf10a8f8","Type":"ContainerDied","Data":"565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e"}
Sep 05 08:01:38.651835 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.651730    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6" event={"ID":"b1eb2a5a-8f15-419b-859d-472fdf10a8f8","Type":"ContainerDied","Data":"da43b3c668379bc7475c746e5af09256fb6fc492e02d819bc91db67ec5c73395"}
Sep 05 08:01:38.651835 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.651751    2569 scope.go:117] "RemoveContainer" containerID="565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e"
Sep 05 08:01:38.662292 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.662271    2569 scope.go:117] "RemoveContainer" containerID="ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6"
Sep 05 08:01:38.671360 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.671345    2569 scope.go:117] "RemoveContainer" containerID="565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e"
Sep 05 08:01:38.671660 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:01:38.671638    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e\": container with ID starting with 565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e not found: ID does not exist" containerID="565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e"
Sep 05 08:01:38.671713 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.671669    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e"} err="failed to get container status \"565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e\": rpc error: code = NotFound desc = could not find container \"565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e\": container with ID starting with 565111b256d1286d5401d1dfbf0c06b67241f7d259532fb7dbc345c19724848e not found: ID does not exist"
Sep 05 08:01:38.671713 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.671686    2569 scope.go:117] "RemoveContainer" containerID="ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6"
Sep 05 08:01:38.671939 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:01:38.671923    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6\": container with ID starting with ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6 not found: ID does not exist" containerID="ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6"
Sep 05 08:01:38.671983 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.671944    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6"} err="failed to get container status \"ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6\": rpc error: code = NotFound desc = could not find container \"ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6\": container with ID starting with ac12334d74efd57276cc1ff421b22904d5ac4bf3ba0ed067e0fc2444360578d6 not found: ID does not exist"
Sep 05 08:01:38.676730 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.676710    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"]
Sep 05 08:01:38.679279 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.679260    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-73e112cd-openshift-default-644d6c49cb-4vxg6"]
Sep 05 08:01:38.716671 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.716643    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwk2d\" (UniqueName: \"kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-kube-api-access-hwk2d\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:38.716671 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.716670    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:38.716862 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.716682    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:38.716862 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:38.716695    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1eb2a5a-8f15-419b-859d-472fdf10a8f8-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:01:39.121166 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:39.121129    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:39.123763 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:39.123735    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls\") pod \"maas-api-e2e-migrate-7dd976de-5464566d84-87zs5\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:39.226799 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:39.226767    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:39.511418 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:39.511344    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"]
Sep 05 08:01:39.514394 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:01:39.514368    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf773c5a_b15d_4131_8ea2_b39dc9c3f6c0.slice/crio-5caf890b3160fd647bc69080e740b50f76bbbf5cd6d1277729b20265b6b1a7fd WatchSource:0}: Error finding container 5caf890b3160fd647bc69080e740b50f76bbbf5cd6d1277729b20265b6b1a7fd: Status 404 returned error can't find the container with id 5caf890b3160fd647bc69080e740b50f76bbbf5cd6d1277729b20265b6b1a7fd
Sep 05 08:01:39.575440 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:39.575404    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" path="/var/lib/kubelet/pods/b1eb2a5a-8f15-419b-859d-472fdf10a8f8/volumes"
Sep 05 08:01:39.658076 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:39.658037    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5" event={"ID":"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0","Type":"ContainerStarted","Data":"5caf890b3160fd647bc69080e740b50f76bbbf5cd6d1277729b20265b6b1a7fd"}
Sep 05 08:01:40.666505 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:40.666471    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5" event={"ID":"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0","Type":"ContainerStarted","Data":"582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2"}
Sep 05 08:01:40.666994 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:40.666590    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:40.689361 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:40.689315    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5" podStartSLOduration=3.333447275 podStartE2EDuration="3.689301536s" totalImagesPullingTime="355.854261ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:01:40.687966816 +0000 UTC m=+781.633288622" watchObservedRunningTime="2026-09-05 08:01:40.689301536 +0000 UTC m=+781.634623344"
Sep 05 08:01:46.677503 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:46.677468    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:01:52.677674 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:52.677634    2569 patch_prober.go:28] interesting pod/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 08:01:52.678087 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:52.677709    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 08:01:54.757726 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:54.757696    2569 generic.go:356] "Generic (PLEG): container finished" podID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerID="04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8" exitCode=0
Sep 05 08:01:54.758142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:54.757756    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" event={"ID":"859f0b10-3b9d-4e97-954b-024b82d3b870","Type":"ContainerDied","Data":"04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8"}
Sep 05 08:01:54.758142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:54.758118    2569 scope.go:117] "RemoveContainer" containerID="04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8"
Sep 05 08:01:54.759268 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:54.759237    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:54.759323 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:54.759311    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:54.759360 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:54.759343    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:01:55.767787 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:55.767750    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" event={"ID":"859f0b10-3b9d-4e97-954b-024b82d3b870","Type":"ContainerStarted","Data":"2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469"}
Sep 05 08:01:56.582023 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:56.581992    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:01:56.582254 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:56.582232    2569 patch_prober.go:28] interesting pod/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 05 08:01:56.582308 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:56.582288    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 05 08:01:57.581394 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:57.581359    2569 patch_prober.go:28] interesting pod/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 05 08:01:57.581804 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:57.581411    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 05 08:01:58.582084 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:58.582049    2569 patch_prober.go:28] interesting pod/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 05 08:01:58.582474 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:58.582110    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 05 08:01:59.582330 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:59.582296    2569 patch_prober.go:28] interesting pod/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 05 08:01:59.582739 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:01:59.582347    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 05 08:02:00.581998 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:00.581906    2569 patch_prober.go:28] interesting pod/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 05 08:02:00.581998 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:00.581971    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 05 08:02:01.586437 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:01.586401    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:02:01.586852 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:01.586809    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:02:01.587424 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:01.587403    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:02:08.804380 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:08.804311    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"]
Sep 05 08:02:08.805670 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:08.805463    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5" podUID="cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0" containerName="maas-api" containerID="cri-o://582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2" gracePeriod=30
Sep 05 08:02:09.063156 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.063095    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:02:09.119924 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.119890    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-openshift-service-ca\") pod \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") "
Sep 05 08:02:09.119924 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.119927    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-metrics-certs\") pod \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") "
Sep 05 08:02:09.120165 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.120010    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-624pz\" (UniqueName: \"kubernetes.io/projected/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-kube-api-access-624pz\") pod \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") "
Sep 05 08:02:09.120165 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.120047    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls\") pod \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\" (UID: \"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0\") "
Sep 05 08:02:09.120326 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.120302    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0" (UID: "cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:02:09.120468 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.120450    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:09.122292 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.122254    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0" (UID: "cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:02:09.122425 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.122338    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-kube-api-access-624pz" (OuterVolumeSpecName: "kube-api-access-624pz") pod "cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0" (UID: "cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0"). InnerVolumeSpecName "kube-api-access-624pz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:02:09.122425 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.122408    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0" (UID: "cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:02:09.223009 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.222934    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-624pz\" (UniqueName: \"kubernetes.io/projected/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-kube-api-access-624pz\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:09.223009 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.222972    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:09.223009 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.222984    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:09.262840 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.262812    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"]
Sep 05 08:02:09.263095 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.263047    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy" containerID="cri-o://2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469" gracePeriod=30
Sep 05 08:02:09.849369 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.849272    2569 generic.go:356] "Generic (PLEG): container finished" podID="cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0" containerID="582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2" exitCode=0
Sep 05 08:02:09.849845 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.849360    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5" event={"ID":"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0","Type":"ContainerDied","Data":"582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2"}
Sep 05 08:02:09.849845 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.849376    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"
Sep 05 08:02:09.849845 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.849412    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5" event={"ID":"cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0","Type":"ContainerDied","Data":"5caf890b3160fd647bc69080e740b50f76bbbf5cd6d1277729b20265b6b1a7fd"}
Sep 05 08:02:09.849845 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.849435    2569 scope.go:117] "RemoveContainer" containerID="582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2"
Sep 05 08:02:09.858740 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.858725    2569 scope.go:117] "RemoveContainer" containerID="582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2"
Sep 05 08:02:09.859012 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:02:09.858994    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2\": container with ID starting with 582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2 not found: ID does not exist" containerID="582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2"
Sep 05 08:02:09.859068 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.859018    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2"} err="failed to get container status \"582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2\": rpc error: code = NotFound desc = could not find container \"582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2\": container with ID starting with 582dd8e80d02902e972ffe507669ce4d3521b92d4e7d10fb3a6408d61ad290f2 not found: ID does not exist"
Sep 05 08:02:09.873064 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.873042    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"]
Sep 05 08:02:09.875555 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:09.875519    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7dd976de-5464566d84-87zs5"]
Sep 05 08:02:10.078190 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.078162    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"]
Sep 05 08:02:10.078632 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.078619    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0" containerName="maas-api"
Sep 05 08:02:10.078691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.078634    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0" containerName="maas-api"
Sep 05 08:02:10.078691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.078643    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy"
Sep 05 08:02:10.078691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.078650    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy"
Sep 05 08:02:10.078691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.078675    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy"
Sep 05 08:02:10.078691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.078681    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy"
Sep 05 08:02:10.078863 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.078755    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0" containerName="maas-api"
Sep 05 08:02:10.078863 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.078764    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy"
Sep 05 08:02:10.078863 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.078773    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1eb2a5a-8f15-419b-859d-472fdf10a8f8" containerName="istio-proxy"
Sep 05 08:02:10.083299 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.083274    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.087622 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.087601    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-29f90b97-openshift-default-dockercfg-wjs2k\""
Sep 05 08:02:10.117318 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.117258    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"]
Sep 05 08:02:10.133814 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.133789    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-data\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.133905 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.133833    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-token\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.133905 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.133891    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-podinfo\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.134024 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.133916    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istiod-ca-cert\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.134082 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.134015    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-envoy\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.134136 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.134081    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-socket\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.134136 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.134111    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-certs\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.134236 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.134154    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp89w\" (UniqueName: \"kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-kube-api-access-tp89w\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.134236 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.134188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-credential-socket\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.235760 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.235727    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-data\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.235927 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.235771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-token\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.235927 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.235813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-podinfo\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.235927 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.235839    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istiod-ca-cert\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.235927 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.235905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-envoy\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.236104 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.235963    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-socket\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.236198 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.236180    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-data\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.236345 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.236311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-certs\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.236396 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.236378    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp89w\" (UniqueName: \"kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-kube-api-access-tp89w\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.236440 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.236406    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-credential-socket\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.236440 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.236328    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-socket\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.236556 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.236506    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-certs\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.236621 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.236596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istiod-ca-cert\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.236736 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.236718    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-credential-socket\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.239036 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.239008    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-envoy\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.239201 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.239181    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-podinfo\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.247396 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.247375    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp89w\" (UniqueName: \"kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-kube-api-access-tp89w\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.249699 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.249682    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-token\") pod \"e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") " pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.393571 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.393457    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:10.681361 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.681337    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"]
Sep 05 08:02:10.683410 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:02:10.683391    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc2a913f_bebd_468b_ad62_c3d297fe9ebd.slice/crio-5bd711a2bcd8348ca4a2564d0f2bc72c6ad3d73bab18fd2b1a1a67aa3a6cb5bc WatchSource:0}: Error finding container 5bd711a2bcd8348ca4a2564d0f2bc72c6ad3d73bab18fd2b1a1a67aa3a6cb5bc: Status 404 returned error can't find the container with id 5bd711a2bcd8348ca4a2564d0f2bc72c6ad3d73bab18fd2b1a1a67aa3a6cb5bc
Sep 05 08:02:10.685816 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.685783    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:02:10.685880 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.685851    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:02:10.685916 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.685880    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:02:10.857500 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.857461    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" event={"ID":"cc2a913f-bebd-468b-ad62-c3d297fe9ebd","Type":"ContainerStarted","Data":"37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f"}
Sep 05 08:02:10.857940 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.857546    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" event={"ID":"cc2a913f-bebd-468b-ad62-c3d297fe9ebd","Type":"ContainerStarted","Data":"5bd711a2bcd8348ca4a2564d0f2bc72c6ad3d73bab18fd2b1a1a67aa3a6cb5bc"}
Sep 05 08:02:10.889997 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:10.889946    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" podStartSLOduration=0.889927471 podStartE2EDuration="889.927471ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:02:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:02:10.887278528 +0000 UTC m=+811.832600343" watchObservedRunningTime="2026-09-05 08:02:10.889927471 +0000 UTC m=+811.835249281"
Sep 05 08:02:11.394270 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:11.394235    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:11.399395 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:11.399375    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:11.572005 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:11.571971    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0" path="/var/lib/kubelet/pods/cf773c5a-b15d-4131-8ea2-b39dc9c3f6c0/volumes"
Sep 05 08:02:11.864322 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:11.864298    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:11.865091 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:11.865072    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:13.458458 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.458421    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"]
Sep 05 08:02:13.462583 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.462564    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.466674 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.466646    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-29f90b97-serving-cert\""
Sep 05 08:02:13.477008 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.476957    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"]
Sep 05 08:02:13.569967 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.569942    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn8wt\" (UniqueName: \"kubernetes.io/projected/74ee5c99-57bc-488e-a67f-d83c6712c84a-kube-api-access-qn8wt\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.570113 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.570011    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee5c99-57bc-488e-a67f-d83c6712c84a-openshift-service-ca\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.570156 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.570134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-metrics-certs\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.570212 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.570198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-maas-api-tls\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.671666 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.671631    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee5c99-57bc-488e-a67f-d83c6712c84a-openshift-service-ca\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.671848 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.671733    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-metrics-certs\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.671930 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.671900    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-maas-api-tls\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.672165 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.672144    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn8wt\" (UniqueName: \"kubernetes.io/projected/74ee5c99-57bc-488e-a67f-d83c6712c84a-kube-api-access-qn8wt\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.672415 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.672390    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee5c99-57bc-488e-a67f-d83c6712c84a-openshift-service-ca\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.674417 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.674362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-metrics-certs\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.674511 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.674453    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-maas-api-tls\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.681736 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.681717    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn8wt\" (UniqueName: \"kubernetes.io/projected/74ee5c99-57bc-488e-a67f-d83c6712c84a-kube-api-access-qn8wt\") pod \"maas-api-e2e-ait-29f90b97-64848c4887-jlfhc\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:13.775822 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:13.775762    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:14.063172 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.063145    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"]
Sep 05 08:02:14.064393 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:02:14.064370    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ee5c99_57bc_488e_a67f_d83c6712c84a.slice/crio-7a0ea2185e5103a07f6d167f86267c2115ad6d98443fd79000b020e09a5dd1f3 WatchSource:0}: Error finding container 7a0ea2185e5103a07f6d167f86267c2115ad6d98443fd79000b020e09a5dd1f3: Status 404 returned error can't find the container with id 7a0ea2185e5103a07f6d167f86267c2115ad6d98443fd79000b020e09a5dd1f3
Sep 05 08:02:14.510334 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.510309    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:02:14.582449 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582426    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tv2r\" (UniqueName: \"kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-kube-api-access-5tv2r\") pod \"859f0b10-3b9d-4e97-954b-024b82d3b870\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") "
Sep 05 08:02:14.582557 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582484    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-certs\") pod \"859f0b10-3b9d-4e97-954b-024b82d3b870\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") "
Sep 05 08:02:14.582557 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582515    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/859f0b10-3b9d-4e97-954b-024b82d3b870-istiod-ca-cert\") pod \"859f0b10-3b9d-4e97-954b-024b82d3b870\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") "
Sep 05 08:02:14.582680 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582576    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-data\") pod \"859f0b10-3b9d-4e97-954b-024b82d3b870\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") "
Sep 05 08:02:14.582680 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582596    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-credential-socket\") pod \"859f0b10-3b9d-4e97-954b-024b82d3b870\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") "
Sep 05 08:02:14.582680 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582642    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-token\") pod \"859f0b10-3b9d-4e97-954b-024b82d3b870\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") "
Sep 05 08:02:14.582888 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582686    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-envoy\") pod \"859f0b10-3b9d-4e97-954b-024b82d3b870\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") "
Sep 05 08:02:14.582888 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582715    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-socket\") pod \"859f0b10-3b9d-4e97-954b-024b82d3b870\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") "
Sep 05 08:02:14.582888 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582741    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-podinfo\") pod \"859f0b10-3b9d-4e97-954b-024b82d3b870\" (UID: \"859f0b10-3b9d-4e97-954b-024b82d3b870\") "
Sep 05 08:02:14.583044 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582882    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-data" (OuterVolumeSpecName: "istio-data") pod "859f0b10-3b9d-4e97-954b-024b82d3b870" (UID: "859f0b10-3b9d-4e97-954b-024b82d3b870"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:02:14.583044 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582896    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "859f0b10-3b9d-4e97-954b-024b82d3b870" (UID: "859f0b10-3b9d-4e97-954b-024b82d3b870"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:02:14.583044 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.582999    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/859f0b10-3b9d-4e97-954b-024b82d3b870-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "859f0b10-3b9d-4e97-954b-024b82d3b870" (UID: "859f0b10-3b9d-4e97-954b-024b82d3b870"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:02:14.583229 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.583208    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:14.583292 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.583236    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:14.583292 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.583251    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/859f0b10-3b9d-4e97-954b-024b82d3b870-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:14.583437 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.583411    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "859f0b10-3b9d-4e97-954b-024b82d3b870" (UID: "859f0b10-3b9d-4e97-954b-024b82d3b870"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:02:14.583595 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.583576    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "859f0b10-3b9d-4e97-954b-024b82d3b870" (UID: "859f0b10-3b9d-4e97-954b-024b82d3b870"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:02:14.584814 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.584790    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-kube-api-access-5tv2r" (OuterVolumeSpecName: "kube-api-access-5tv2r") pod "859f0b10-3b9d-4e97-954b-024b82d3b870" (UID: "859f0b10-3b9d-4e97-954b-024b82d3b870"). InnerVolumeSpecName "kube-api-access-5tv2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:02:14.585364 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.585341    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-token" (OuterVolumeSpecName: "istio-token") pod "859f0b10-3b9d-4e97-954b-024b82d3b870" (UID: "859f0b10-3b9d-4e97-954b-024b82d3b870"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:02:14.585481 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.585465    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "859f0b10-3b9d-4e97-954b-024b82d3b870" (UID: "859f0b10-3b9d-4e97-954b-024b82d3b870"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:02:14.585542 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.585478    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "859f0b10-3b9d-4e97-954b-024b82d3b870" (UID: "859f0b10-3b9d-4e97-954b-024b82d3b870"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:02:14.684567 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.684510    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:14.684682 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.684574    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:14.684682 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.684590    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:14.684682 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.684604    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tv2r\" (UniqueName: \"kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-kube-api-access-5tv2r\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:14.684682 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.684618    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/859f0b10-3b9d-4e97-954b-024b82d3b870-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:14.684682 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.684634    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/859f0b10-3b9d-4e97-954b-024b82d3b870-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:14.884003 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.883967    2569 generic.go:356] "Generic (PLEG): container finished" podID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerID="2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469" exitCode=0
Sep 05 08:02:14.884166 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.884018    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"
Sep 05 08:02:14.884166 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.884053    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" event={"ID":"859f0b10-3b9d-4e97-954b-024b82d3b870","Type":"ContainerDied","Data":"2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469"}
Sep 05 08:02:14.884166 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.884111    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj" event={"ID":"859f0b10-3b9d-4e97-954b-024b82d3b870","Type":"ContainerDied","Data":"7e8781530355b75157ae57a0243c15d84c74769dd9615bf066338ed03847e1b3"}
Sep 05 08:02:14.884166 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.884144    2569 scope.go:117] "RemoveContainer" containerID="2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469"
Sep 05 08:02:14.886348 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.886318    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc" event={"ID":"74ee5c99-57bc-488e-a67f-d83c6712c84a","Type":"ContainerStarted","Data":"b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c"}
Sep 05 08:02:14.886463 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.886390    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc" event={"ID":"74ee5c99-57bc-488e-a67f-d83c6712c84a","Type":"ContainerStarted","Data":"7a0ea2185e5103a07f6d167f86267c2115ad6d98443fd79000b020e09a5dd1f3"}
Sep 05 08:02:14.886957 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.886940    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:14.896213 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.896199    2569 scope.go:117] "RemoveContainer" containerID="04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8"
Sep 05 08:02:14.904962 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.904944    2569 scope.go:117] "RemoveContainer" containerID="2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469"
Sep 05 08:02:14.905240 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:02:14.905223    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469\": container with ID starting with 2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469 not found: ID does not exist" containerID="2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469"
Sep 05 08:02:14.905296 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.905248    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469"} err="failed to get container status \"2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469\": rpc error: code = NotFound desc = could not find container \"2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469\": container with ID starting with 2154dd4078429770cf057d7ebcbe402ff9b66daf3284ff05629f01fcdc05a469 not found: ID does not exist"
Sep 05 08:02:14.905296 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.905263    2569 scope.go:117] "RemoveContainer" containerID="04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8"
Sep 05 08:02:14.905513 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:02:14.905495    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8\": container with ID starting with 04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8 not found: ID does not exist" containerID="04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8"
Sep 05 08:02:14.905571 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.905539    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8"} err="failed to get container status \"04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8\": rpc error: code = NotFound desc = could not find container \"04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8\": container with ID starting with 04d37bb0f19ce6b58f8583fae028c0267bdf3c26ece599a80a7e6145c84d82b8 not found: ID does not exist"
Sep 05 08:02:14.912117 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.912079    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc" podStartSLOduration=1.433446566 podStartE2EDuration="1.912067652s" totalImagesPullingTime="478.621086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:02:14.91028187 +0000 UTC m=+815.855603677" watchObservedRunningTime="2026-09-05 08:02:14.912067652 +0000 UTC m=+815.857389457"
Sep 05 08:02:14.928333 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.928307    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"]
Sep 05 08:02:14.936773 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:14.936726    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-7dd976de-gw-openshift-default-7cbbc5cd8-gf2gj"]
Sep 05 08:02:15.571173 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:15.571143    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" path="/var/lib/kubelet/pods/859f0b10-3b9d-4e97-954b-024b82d3b870/volumes"
Sep 05 08:02:20.898353 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:20.898324    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:35.016923 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:35.016884    2569 generic.go:356] "Generic (PLEG): container finished" podID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerID="37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f" exitCode=0
Sep 05 08:02:35.017394 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:35.016948    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" event={"ID":"cc2a913f-bebd-468b-ad62-c3d297fe9ebd","Type":"ContainerDied","Data":"37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f"}
Sep 05 08:02:35.017394 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:35.017374    2569 scope.go:117] "RemoveContainer" containerID="37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f"
Sep 05 08:02:35.018737 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:35.018705    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:02:35.018820 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:35.018773    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:02:35.018820 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:35.018802    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:02:36.024349 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:36.024318    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" event={"ID":"cc2a913f-bebd-468b-ad62-c3d297fe9ebd","Type":"ContainerStarted","Data":"90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899"}
Sep 05 08:02:36.394511 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:36.394479    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:36.394767 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:36.394745    2569 patch_prober.go:28] interesting pod/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 05 08:02:36.394815 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:36.394799    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 05 08:02:37.394449 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:37.394419    2569 patch_prober.go:28] interesting pod/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 05 08:02:37.394858 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:37.394468    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 05 08:02:38.394327 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:38.394295    2569 patch_prober.go:28] interesting pod/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 05 08:02:38.394552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:38.394345    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 05 08:02:39.394136 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:39.394103    2569 patch_prober.go:28] interesting pod/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 05 08:02:39.394331 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:39.394154    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 05 08:02:40.394172 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:40.394140    2569 patch_prober.go:28] interesting pod/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 05 08:02:40.394604 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:40.394201    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 05 08:02:41.435465 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:41.435423    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:41.436079 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:41.436039    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:41.436866 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:41.436846    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:44.009412 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.009377    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"]
Sep 05 08:02:44.009925 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.009642    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc" podUID="74ee5c99-57bc-488e-a67f-d83c6712c84a" containerName="maas-api" containerID="cri-o://b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c" gracePeriod=30
Sep 05 08:02:44.271552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.271472    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:44.411497 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.411458    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-metrics-certs\") pod \"74ee5c99-57bc-488e-a67f-d83c6712c84a\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") "
Sep 05 08:02:44.411695 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.411516    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee5c99-57bc-488e-a67f-d83c6712c84a-openshift-service-ca\") pod \"74ee5c99-57bc-488e-a67f-d83c6712c84a\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") "
Sep 05 08:02:44.411695 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.411589    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-maas-api-tls\") pod \"74ee5c99-57bc-488e-a67f-d83c6712c84a\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") "
Sep 05 08:02:44.411695 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.411637    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn8wt\" (UniqueName: \"kubernetes.io/projected/74ee5c99-57bc-488e-a67f-d83c6712c84a-kube-api-access-qn8wt\") pod \"74ee5c99-57bc-488e-a67f-d83c6712c84a\" (UID: \"74ee5c99-57bc-488e-a67f-d83c6712c84a\") "
Sep 05 08:02:44.412166 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.412136    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ee5c99-57bc-488e-a67f-d83c6712c84a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "74ee5c99-57bc-488e-a67f-d83c6712c84a" (UID: "74ee5c99-57bc-488e-a67f-d83c6712c84a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:02:44.414784 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.414745    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "74ee5c99-57bc-488e-a67f-d83c6712c84a" (UID: "74ee5c99-57bc-488e-a67f-d83c6712c84a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:02:44.414784 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.414757    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74ee5c99-57bc-488e-a67f-d83c6712c84a-kube-api-access-qn8wt" (OuterVolumeSpecName: "kube-api-access-qn8wt") pod "74ee5c99-57bc-488e-a67f-d83c6712c84a" (UID: "74ee5c99-57bc-488e-a67f-d83c6712c84a"). InnerVolumeSpecName "kube-api-access-qn8wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:02:44.416400 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.416368    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "74ee5c99-57bc-488e-a67f-d83c6712c84a" (UID: "74ee5c99-57bc-488e-a67f-d83c6712c84a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:02:44.512962 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.512934    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee5c99-57bc-488e-a67f-d83c6712c84a-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:44.512962 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.512963    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:44.513151 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.512974    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn8wt\" (UniqueName: \"kubernetes.io/projected/74ee5c99-57bc-488e-a67f-d83c6712c84a-kube-api-access-qn8wt\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:44.513151 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:44.512982    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74ee5c99-57bc-488e-a67f-d83c6712c84a-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:45.101887 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.101850    2569 generic.go:356] "Generic (PLEG): container finished" podID="74ee5c99-57bc-488e-a67f-d83c6712c84a" containerID="b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c" exitCode=0
Sep 05 08:02:45.102283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.101896    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"
Sep 05 08:02:45.102283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.101941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc" event={"ID":"74ee5c99-57bc-488e-a67f-d83c6712c84a","Type":"ContainerDied","Data":"b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c"}
Sep 05 08:02:45.102283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.102000    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc" event={"ID":"74ee5c99-57bc-488e-a67f-d83c6712c84a","Type":"ContainerDied","Data":"7a0ea2185e5103a07f6d167f86267c2115ad6d98443fd79000b020e09a5dd1f3"}
Sep 05 08:02:45.102283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.102031    2569 scope.go:117] "RemoveContainer" containerID="b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c"
Sep 05 08:02:45.111725 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.111710    2569 scope.go:117] "RemoveContainer" containerID="b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c"
Sep 05 08:02:45.112014 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:02:45.111995    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c\": container with ID starting with b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c not found: ID does not exist" containerID="b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c"
Sep 05 08:02:45.112106 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.112023    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c"} err="failed to get container status \"b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c\": rpc error: code = NotFound desc = could not find container \"b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c\": container with ID starting with b5171103120a1b6d8e39176c109344dcaa1e75765392103308e878606de4254c not found: ID does not exist"
Sep 05 08:02:45.129306 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.129269    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"]
Sep 05 08:02:45.131210 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.131187    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29f90b97-64848c4887-jlfhc"]
Sep 05 08:02:45.445590 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.445495    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"]
Sep 05 08:02:45.445802 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.445765    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy" containerID="cri-o://90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899" gracePeriod=30
Sep 05 08:02:45.571244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:45.571211    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74ee5c99-57bc-488e-a67f-d83c6712c84a" path="/var/lib/kubelet/pods/74ee5c99-57bc-488e-a67f-d83c6712c84a/volumes"
Sep 05 08:02:46.248896 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.248865    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"]
Sep 05 08:02:46.249367 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.249353    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy"
Sep 05 08:02:46.249421 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.249369    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy"
Sep 05 08:02:46.249421 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.249377    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ee5c99-57bc-488e-a67f-d83c6712c84a" containerName="maas-api"
Sep 05 08:02:46.249421 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.249384    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ee5c99-57bc-488e-a67f-d83c6712c84a" containerName="maas-api"
Sep 05 08:02:46.249421 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.249406    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy"
Sep 05 08:02:46.249421 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.249412    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy"
Sep 05 08:02:46.249620 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.249481    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy"
Sep 05 08:02:46.249620 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.249489    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="859f0b10-3b9d-4e97-954b-024b82d3b870" containerName="istio-proxy"
Sep 05 08:02:46.249620 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.249497    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="74ee5c99-57bc-488e-a67f-d83c6712c84a" containerName="maas-api"
Sep 05 08:02:46.252204 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.252187    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.258218 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.258197    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-46865726-openshift-default-dockercfg-dxhhk\""
Sep 05 08:02:46.267790 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.267769    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"]
Sep 05 08:02:46.331855 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.331831    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-envoy\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.331995 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.331886    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-credential-socket\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.331995 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.331928    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-socket\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.331995 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.331961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-data\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.332118 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.332019    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-token\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.332118 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.332075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-certs\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.332182 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.332121    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istiod-ca-cert\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.332182 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.332143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9865p\" (UniqueName: \"kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-kube-api-access-9865p\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.332182 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.332169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-podinfo\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.433084 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433050    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-token\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.433253 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433126    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-certs\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.433253 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433186    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istiod-ca-cert\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.433253 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433221    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9865p\" (UniqueName: \"kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-kube-api-access-9865p\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.433423 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-podinfo\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.433423 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433376    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-envoy\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.434092 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433431    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-credential-socket\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.434092 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433581    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-socket\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.434092 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-data\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.434092 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433800    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-certs\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.434092 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.433997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-credential-socket\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.434405 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.434117    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-data\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.434405 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.434139    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istiod-ca-cert\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.434405 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.434163    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-socket\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.436878 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.436853    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-envoy\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.436971 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.436938    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-podinfo\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.442401 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.442378    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-token\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.447354 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.447311    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9865p\" (UniqueName: \"kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-kube-api-access-9865p\") pod \"e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") " pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.563000 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.562923    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:46.840705 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.840659    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"]
Sep 05 08:02:46.842742 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:02:46.842712    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee80e01_bf85_4b0e_bf3f_7282c5c2d377.slice/crio-10379b5cc4a5c017c77c2cd8f8adfce3a4532b09c33860118d570b90b5dd8f91 WatchSource:0}: Error finding container 10379b5cc4a5c017c77c2cd8f8adfce3a4532b09c33860118d570b90b5dd8f91: Status 404 returned error can't find the container with id 10379b5cc4a5c017c77c2cd8f8adfce3a4532b09c33860118d570b90b5dd8f91
Sep 05 08:02:46.845936 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.845904    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:02:46.846029 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.845974    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:02:46.846029 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:46.846003    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:02:47.116798 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:47.116707    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" event={"ID":"bee80e01-bf85-4b0e-bf3f-7282c5c2d377","Type":"ContainerStarted","Data":"f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553"}
Sep 05 08:02:47.116798 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:47.116759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" event={"ID":"bee80e01-bf85-4b0e-bf3f-7282c5c2d377","Type":"ContainerStarted","Data":"10379b5cc4a5c017c77c2cd8f8adfce3a4532b09c33860118d570b90b5dd8f91"}
Sep 05 08:02:47.143397 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:47.143321    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" podStartSLOduration=1.143302727 podStartE2EDuration="1.143302727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:02:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:02:47.140677988 +0000 UTC m=+848.085999794" watchObservedRunningTime="2026-09-05 08:02:47.143302727 +0000 UTC m=+848.088624536"
Sep 05 08:02:47.563391 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:47.563312    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:47.571887 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:47.571866    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:48.122489 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:48.122461    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:48.123488 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:48.123468    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:02:49.816120 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:49.816089    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"]
Sep 05 08:02:49.818745 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:49.818726    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:49.821513 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:49.821480    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-46865726-serving-cert\""
Sep 05 08:02:49.832410 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:49.832389    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"]
Sep 05 08:02:49.968759 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:49.968722    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cbxj\" (UniqueName: \"kubernetes.io/projected/8595634f-0619-4911-a7cf-2328f4774a9f-kube-api-access-8cbxj\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:49.968759 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:49.968761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8595634f-0619-4911-a7cf-2328f4774a9f-openshift-service-ca\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:49.968962 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:49.968791    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-metrics-certs\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:49.969012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:49.968996    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-maas-api-tls\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.069751 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.069712    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-maas-api-tls\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.069953 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.069776    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cbxj\" (UniqueName: \"kubernetes.io/projected/8595634f-0619-4911-a7cf-2328f4774a9f-kube-api-access-8cbxj\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.069953 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.069809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8595634f-0619-4911-a7cf-2328f4774a9f-openshift-service-ca\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.069953 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.069845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-metrics-certs\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.069953 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:02:50.069893    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-46865726-serving-cert: secret "maas-api-e2e-derive-46865726-serving-cert" not found
Sep 05 08:02:50.070168 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:02:50.069967    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-maas-api-tls podName:8595634f-0619-4911-a7cf-2328f4774a9f nodeName:}" failed. No retries permitted until 2026-09-05 08:02:50.56994363 +0000 UTC m=+851.515265429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-maas-api-tls") pod "maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8" (UID: "8595634f-0619-4911-a7cf-2328f4774a9f") : secret "maas-api-e2e-derive-46865726-serving-cert" not found
Sep 05 08:02:50.070890 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.070869    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8595634f-0619-4911-a7cf-2328f4774a9f-openshift-service-ca\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.072883 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.072862    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-metrics-certs\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.093143 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.093119    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cbxj\" (UniqueName: \"kubernetes.io/projected/8595634f-0619-4911-a7cf-2328f4774a9f-kube-api-access-8cbxj\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.575100 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.575064    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-maas-api-tls\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.577423 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.577401    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-maas-api-tls\") pod \"maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.705223 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.705200    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:50.732705 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.732642    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:50.878300 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.878269    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-token\") pod \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") "
Sep 05 08:02:50.878300 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.878302    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istiod-ca-cert\") pod \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") "
Sep 05 08:02:50.878850 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.878326    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp89w\" (UniqueName: \"kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-kube-api-access-tp89w\") pod \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") "
Sep 05 08:02:50.878850 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.878369    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-certs\") pod \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") "
Sep 05 08:02:50.878850 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.878429    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-credential-socket\") pod \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") "
Sep 05 08:02:50.878850 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.878482    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-socket\") pod \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") "
Sep 05 08:02:50.878850 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.878509    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-data\") pod \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") "
Sep 05 08:02:50.878850 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.878552    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-podinfo\") pod \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") "
Sep 05 08:02:50.878850 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.878719    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-envoy\") pod \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\" (UID: \"cc2a913f-bebd-468b-ad62-c3d297fe9ebd\") "
Sep 05 08:02:50.879255 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.878878    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cc2a913f-bebd-468b-ad62-c3d297fe9ebd" (UID: "cc2a913f-bebd-468b-ad62-c3d297fe9ebd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:02:50.879255 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.879018    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cc2a913f-bebd-468b-ad62-c3d297fe9ebd" (UID: "cc2a913f-bebd-468b-ad62-c3d297fe9ebd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:02:50.879255 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.879034    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-data" (OuterVolumeSpecName: "istio-data") pod "cc2a913f-bebd-468b-ad62-c3d297fe9ebd" (UID: "cc2a913f-bebd-468b-ad62-c3d297fe9ebd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:02:50.879255 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.879183    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:50.879255 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.879201    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:50.879255 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.879214    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:50.879643 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.879266    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cc2a913f-bebd-468b-ad62-c3d297fe9ebd" (UID: "cc2a913f-bebd-468b-ad62-c3d297fe9ebd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:02:50.879643 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.879320    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cc2a913f-bebd-468b-ad62-c3d297fe9ebd" (UID: "cc2a913f-bebd-468b-ad62-c3d297fe9ebd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:02:50.881474 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.881413    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-token" (OuterVolumeSpecName: "istio-token") pod "cc2a913f-bebd-468b-ad62-c3d297fe9ebd" (UID: "cc2a913f-bebd-468b-ad62-c3d297fe9ebd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:02:50.881474 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.881454    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-kube-api-access-tp89w" (OuterVolumeSpecName: "kube-api-access-tp89w") pod "cc2a913f-bebd-468b-ad62-c3d297fe9ebd" (UID: "cc2a913f-bebd-468b-ad62-c3d297fe9ebd"). InnerVolumeSpecName "kube-api-access-tp89w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:02:50.881810 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.881680    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cc2a913f-bebd-468b-ad62-c3d297fe9ebd" (UID: "cc2a913f-bebd-468b-ad62-c3d297fe9ebd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:02:50.882187 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.882157    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cc2a913f-bebd-468b-ad62-c3d297fe9ebd" (UID: "cc2a913f-bebd-468b-ad62-c3d297fe9ebd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:02:50.980211 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.980178    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:50.980211 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.980205    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:50.980211 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.980215    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp89w\" (UniqueName: \"kubernetes.io/projected/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-kube-api-access-tp89w\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:50.980493 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.980235    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:50.980493 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.980247    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:50.980493 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:50.980255    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc2a913f-bebd-468b-ad62-c3d297fe9ebd-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:02:51.025779 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.025754    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"]
Sep 05 08:02:51.027291 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:02:51.027262    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8595634f_0619_4911_a7cf_2328f4774a9f.slice/crio-d6162df1c4c62c02ab2d4ee03c4fa179ec7ec58861d59501942bb9820b881835 WatchSource:0}: Error finding container d6162df1c4c62c02ab2d4ee03c4fa179ec7ec58861d59501942bb9820b881835: Status 404 returned error can't find the container with id d6162df1c4c62c02ab2d4ee03c4fa179ec7ec58861d59501942bb9820b881835
Sep 05 08:02:51.143854 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.143826    2569 generic.go:356] "Generic (PLEG): container finished" podID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerID="90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899" exitCode=0
Sep 05 08:02:51.144025 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.143888    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"
Sep 05 08:02:51.144025 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.143910    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" event={"ID":"cc2a913f-bebd-468b-ad62-c3d297fe9ebd","Type":"ContainerDied","Data":"90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899"}
Sep 05 08:02:51.144025 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.143998    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2" event={"ID":"cc2a913f-bebd-468b-ad62-c3d297fe9ebd","Type":"ContainerDied","Data":"5bd711a2bcd8348ca4a2564d0f2bc72c6ad3d73bab18fd2b1a1a67aa3a6cb5bc"}
Sep 05 08:02:51.144195 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.144048    2569 scope.go:117] "RemoveContainer" containerID="90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899"
Sep 05 08:02:51.146565 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.146517    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8" event={"ID":"8595634f-0619-4911-a7cf-2328f4774a9f","Type":"ContainerStarted","Data":"d6162df1c4c62c02ab2d4ee03c4fa179ec7ec58861d59501942bb9820b881835"}
Sep 05 08:02:51.156794 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.156775    2569 scope.go:117] "RemoveContainer" containerID="37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f"
Sep 05 08:02:51.166954 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.166935    2569 scope.go:117] "RemoveContainer" containerID="90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899"
Sep 05 08:02:51.167309 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:02:51.167270    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899\": container with ID starting with 90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899 not found: ID does not exist" containerID="90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899"
Sep 05 08:02:51.167422 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.167307    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899"} err="failed to get container status \"90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899\": rpc error: code = NotFound desc = could not find container \"90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899\": container with ID starting with 90491696bd88e36259228f0c6fd98a25f77dd73bb2dbc6284d8fac55d9165899 not found: ID does not exist"
Sep 05 08:02:51.167422 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.167328    2569 scope.go:117] "RemoveContainer" containerID="37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f"
Sep 05 08:02:51.167743 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:02:51.167675    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f\": container with ID starting with 37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f not found: ID does not exist" containerID="37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f"
Sep 05 08:02:51.167743 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.167705    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f"} err="failed to get container status \"37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f\": rpc error: code = NotFound desc = could not find container \"37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f\": container with ID starting with 37d483bf895468272a01c6b4fc393aa849dee1ef77c8e241fb0eecd5a61ad71f not found: ID does not exist"
Sep 05 08:02:51.171834 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.171812    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"]
Sep 05 08:02:51.177159 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.177140    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-29f90b97-openshift-default-5d887486bd-2ssz2"]
Sep 05 08:02:51.570969 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:51.570940    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" path="/var/lib/kubelet/pods/cc2a913f-bebd-468b-ad62-c3d297fe9ebd/volumes"
Sep 05 08:02:52.161919 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:52.161887    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8" event={"ID":"8595634f-0619-4911-a7cf-2328f4774a9f","Type":"ContainerStarted","Data":"1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7"}
Sep 05 08:02:52.162382 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:52.162074    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:02:52.184698 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:52.184646    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8" podStartSLOduration=2.6352138849999998 podStartE2EDuration="3.184628627s" totalImagesPullingTime="549.414742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:02:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:02:52.181381254 +0000 UTC m=+853.126703061" watchObservedRunningTime="2026-09-05 08:02:52.184628627 +0000 UTC m=+853.129950432"
Sep 05 08:02:58.177030 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:02:58.176999    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:03:03.635512 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.635481    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"]
Sep 05 08:03:03.635957 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.635716    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf" podUID="f1ed0ff2-2dfc-4dc0-af8d-549c299bc293" containerName="maas-api" containerID="cri-o://c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99" gracePeriod=30
Sep 05 08:03:03.936483 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.936448    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:03:03.993951 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.993914    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-openshift-service-ca\") pod \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") "
Sep 05 08:03:03.994121 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.994017    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgmj7\" (UniqueName: \"kubernetes.io/projected/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-kube-api-access-sgmj7\") pod \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") "
Sep 05 08:03:03.994121 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.994058    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-maas-api-tls\") pod \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") "
Sep 05 08:03:03.994238 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.994122    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-metrics-certs\") pod \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\" (UID: \"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293\") "
Sep 05 08:03:03.994238 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.994219    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1ed0ff2-2dfc-4dc0-af8d-549c299bc293" (UID: "f1ed0ff2-2dfc-4dc0-af8d-549c299bc293"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:03:03.994567 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.994546    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:03.996247 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.996210    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-kube-api-access-sgmj7" (OuterVolumeSpecName: "kube-api-access-sgmj7") pod "f1ed0ff2-2dfc-4dc0-af8d-549c299bc293" (UID: "f1ed0ff2-2dfc-4dc0-af8d-549c299bc293"). InnerVolumeSpecName "kube-api-access-sgmj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:03:03.996349 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.996300    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1ed0ff2-2dfc-4dc0-af8d-549c299bc293" (UID: "f1ed0ff2-2dfc-4dc0-af8d-549c299bc293"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:03:03.996349 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:03.996318    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f1ed0ff2-2dfc-4dc0-af8d-549c299bc293" (UID: "f1ed0ff2-2dfc-4dc0-af8d-549c299bc293"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:03:04.095580 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.095550    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgmj7\" (UniqueName: \"kubernetes.io/projected/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-kube-api-access-sgmj7\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:04.095580 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.095579    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:04.095580 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.095590    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:04.232125 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.232032    2569 generic.go:356] "Generic (PLEG): container finished" podID="f1ed0ff2-2dfc-4dc0-af8d-549c299bc293" containerID="c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99" exitCode=0
Sep 05 08:03:04.232125 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.232084    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"
Sep 05 08:03:04.232125 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.232116    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf" event={"ID":"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293","Type":"ContainerDied","Data":"c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99"}
Sep 05 08:03:04.232398 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.232174    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf" event={"ID":"f1ed0ff2-2dfc-4dc0-af8d-549c299bc293","Type":"ContainerDied","Data":"5a7d34a7584eae82a709bf8bb72b77a073e93443c4b111d26d955fc69c9969c5"}
Sep 05 08:03:04.232398 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.232198    2569 scope.go:117] "RemoveContainer" containerID="c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99"
Sep 05 08:03:04.251789 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.251763    2569 scope.go:117] "RemoveContainer" containerID="c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99"
Sep 05 08:03:04.252115 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:03:04.252084    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99\": container with ID starting with c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99 not found: ID does not exist" containerID="c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99"
Sep 05 08:03:04.252198 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.252127    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99"} err="failed to get container status \"c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99\": rpc error: code = NotFound desc = could not find container \"c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99\": container with ID starting with c23bd2433874e653c5db50d2fc8ec77fb79552e20843e8cb531d1cd5e088eb99 not found: ID does not exist"
Sep 05 08:03:04.272100 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.272057    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"]
Sep 05 08:03:04.272747 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:04.272724    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4faf97-5ffc675b8c-rdkgf"]
Sep 05 08:03:05.571485 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:05.571452    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ed0ff2-2dfc-4dc0-af8d-549c299bc293" path="/var/lib/kubelet/pods/f1ed0ff2-2dfc-4dc0-af8d-549c299bc293/volumes"
Sep 05 08:03:11.277251 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:11.277216    2569 generic.go:356] "Generic (PLEG): container finished" podID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerID="f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553" exitCode=0
Sep 05 08:03:11.277700 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:11.277291    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" event={"ID":"bee80e01-bf85-4b0e-bf3f-7282c5c2d377","Type":"ContainerDied","Data":"f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553"}
Sep 05 08:03:11.277700 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:11.277662    2569 scope.go:117] "RemoveContainer" containerID="f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553"
Sep 05 08:03:11.278731 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:11.278701    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:03:11.278845 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:11.278779    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:03:11.278845 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:11.278826    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:03:12.284729 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:12.284686    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" event={"ID":"bee80e01-bf85-4b0e-bf3f-7282c5c2d377","Type":"ContainerStarted","Data":"f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79"}
Sep 05 08:03:12.564029 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:12.563933    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:03:12.564280 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:12.564252    2569 patch_prober.go:28] interesting pod/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 05 08:03:12.564375 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:12.564328    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 05 08:03:13.564118 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:13.564082    2569 patch_prober.go:28] interesting pod/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 05 08:03:13.567510 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:13.567477    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 05 08:03:14.563775 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:14.563730    2569 patch_prober.go:28] interesting pod/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 05 08:03:14.564014 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:14.563796    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 05 08:03:15.563639 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:15.563602    2569 patch_prober.go:28] interesting pod/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 05 08:03:15.564104 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:15.563663    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 05 08:03:16.563506 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:16.563456    2569 patch_prober.go:28] interesting pod/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 05 08:03:16.563736 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:16.563548    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 05 08:03:17.655846 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:17.655815    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:03:17.656253 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:17.656234    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:03:17.657221 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:17.657201    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:03:20.376142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.376107    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"]
Sep 05 08:03:20.376563 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.376321    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8" podUID="8595634f-0619-4911-a7cf-2328f4774a9f" containerName="maas-api" containerID="cri-o://1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7" gracePeriod=30
Sep 05 08:03:20.637277 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.637217    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:03:20.786503 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.786464    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"]
Sep 05 08:03:20.786809 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.786773    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy" containerID="cri-o://f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79" gracePeriod=30
Sep 05 08:03:20.789654 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.789627    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cbxj\" (UniqueName: \"kubernetes.io/projected/8595634f-0619-4911-a7cf-2328f4774a9f-kube-api-access-8cbxj\") pod \"8595634f-0619-4911-a7cf-2328f4774a9f\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") "
Sep 05 08:03:20.789808 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.789719    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-maas-api-tls\") pod \"8595634f-0619-4911-a7cf-2328f4774a9f\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") "
Sep 05 08:03:20.789808 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.789799    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8595634f-0619-4911-a7cf-2328f4774a9f-openshift-service-ca\") pod \"8595634f-0619-4911-a7cf-2328f4774a9f\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") "
Sep 05 08:03:20.789905 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.789852    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-metrics-certs\") pod \"8595634f-0619-4911-a7cf-2328f4774a9f\" (UID: \"8595634f-0619-4911-a7cf-2328f4774a9f\") "
Sep 05 08:03:20.792462 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.792417    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8595634f-0619-4911-a7cf-2328f4774a9f-kube-api-access-8cbxj" (OuterVolumeSpecName: "kube-api-access-8cbxj") pod "8595634f-0619-4911-a7cf-2328f4774a9f" (UID: "8595634f-0619-4911-a7cf-2328f4774a9f"). InnerVolumeSpecName "kube-api-access-8cbxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:03:20.792674 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.792646    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8595634f-0619-4911-a7cf-2328f4774a9f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8595634f-0619-4911-a7cf-2328f4774a9f" (UID: "8595634f-0619-4911-a7cf-2328f4774a9f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:03:20.798953 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.798919    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8595634f-0619-4911-a7cf-2328f4774a9f" (UID: "8595634f-0619-4911-a7cf-2328f4774a9f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:03:20.804613 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.804586    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8595634f-0619-4911-a7cf-2328f4774a9f" (UID: "8595634f-0619-4911-a7cf-2328f4774a9f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:03:20.891723 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.891624    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8595634f-0619-4911-a7cf-2328f4774a9f-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:20.891723 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.891656    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:20.891723 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.891669    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cbxj\" (UniqueName: \"kubernetes.io/projected/8595634f-0619-4911-a7cf-2328f4774a9f-kube-api-access-8cbxj\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:20.891723 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:20.891681    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8595634f-0619-4911-a7cf-2328f4774a9f-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:21.333808 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:21.333774    2569 generic.go:356] "Generic (PLEG): container finished" podID="8595634f-0619-4911-a7cf-2328f4774a9f" containerID="1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7" exitCode=0
Sep 05 08:03:21.333976 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:21.333826    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8" event={"ID":"8595634f-0619-4911-a7cf-2328f4774a9f","Type":"ContainerDied","Data":"1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7"}
Sep 05 08:03:21.333976 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:21.333840    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"
Sep 05 08:03:21.333976 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:21.333893    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8" event={"ID":"8595634f-0619-4911-a7cf-2328f4774a9f","Type":"ContainerDied","Data":"d6162df1c4c62c02ab2d4ee03c4fa179ec7ec58861d59501942bb9820b881835"}
Sep 05 08:03:21.333976 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:21.333924    2569 scope.go:117] "RemoveContainer" containerID="1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7"
Sep 05 08:03:21.344442 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:21.344422    2569 scope.go:117] "RemoveContainer" containerID="1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7"
Sep 05 08:03:21.344742 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:03:21.344722    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7\": container with ID starting with 1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7 not found: ID does not exist" containerID="1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7"
Sep 05 08:03:21.344802 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:21.344750    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7"} err="failed to get container status \"1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7\": rpc error: code = NotFound desc = could not find container \"1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7\": container with ID starting with 1a82ed9a9d415d6621b1aaf6c760e249a1b1ec0fa04bb2e65252d066324139e7 not found: ID does not exist"
Sep 05 08:03:21.358337 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:21.358315    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"]
Sep 05 08:03:21.366436 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:21.366413    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-46865726-5df47ff4c6-zmgp8"]
Sep 05 08:03:21.570973 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:21.570938    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8595634f-0619-4911-a7cf-2328f4774a9f" path="/var/lib/kubelet/pods/8595634f-0619-4911-a7cf-2328f4774a9f/volumes"
Sep 05 08:03:26.048425 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.048367    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:03:26.145367 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145337    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-socket\") pod \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") "
Sep 05 08:03:26.145367 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145373    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-certs\") pod \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") "
Sep 05 08:03:26.145640 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145407    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-token\") pod \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") "
Sep 05 08:03:26.145640 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145442    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-credential-socket\") pod \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") "
Sep 05 08:03:26.145640 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145457    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9865p\" (UniqueName: \"kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-kube-api-access-9865p\") pod \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") "
Sep 05 08:03:26.145640 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145494    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-podinfo\") pod \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") "
Sep 05 08:03:26.145640 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145513    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-data\") pod \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") "
Sep 05 08:03:26.145640 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145579    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-envoy\") pod \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") "
Sep 05 08:03:26.145640 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145626    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istiod-ca-cert\") pod \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\" (UID: \"bee80e01-bf85-4b0e-bf3f-7282c5c2d377\") "
Sep 05 08:03:26.145981 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145699    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bee80e01-bf85-4b0e-bf3f-7282c5c2d377" (UID: "bee80e01-bf85-4b0e-bf3f-7282c5c2d377"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:03:26.145981 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.145862    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bee80e01-bf85-4b0e-bf3f-7282c5c2d377" (UID: "bee80e01-bf85-4b0e-bf3f-7282c5c2d377"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:03:26.146087 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.146045    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:26.146087 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.146047    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bee80e01-bf85-4b0e-bf3f-7282c5c2d377" (UID: "bee80e01-bf85-4b0e-bf3f-7282c5c2d377"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:03:26.146087 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.146065    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:26.146087 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.146062    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-data" (OuterVolumeSpecName: "istio-data") pod "bee80e01-bf85-4b0e-bf3f-7282c5c2d377" (UID: "bee80e01-bf85-4b0e-bf3f-7282c5c2d377"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:03:26.146305 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.146094    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bee80e01-bf85-4b0e-bf3f-7282c5c2d377" (UID: "bee80e01-bf85-4b0e-bf3f-7282c5c2d377"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:03:26.147978 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.147948    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-kube-api-access-9865p" (OuterVolumeSpecName: "kube-api-access-9865p") pod "bee80e01-bf85-4b0e-bf3f-7282c5c2d377" (UID: "bee80e01-bf85-4b0e-bf3f-7282c5c2d377"). InnerVolumeSpecName "kube-api-access-9865p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:03:26.147978 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.147963    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bee80e01-bf85-4b0e-bf3f-7282c5c2d377" (UID: "bee80e01-bf85-4b0e-bf3f-7282c5c2d377"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:03:26.148129 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.147995    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-token" (OuterVolumeSpecName: "istio-token") pod "bee80e01-bf85-4b0e-bf3f-7282c5c2d377" (UID: "bee80e01-bf85-4b0e-bf3f-7282c5c2d377"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:03:26.148129 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.148019    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bee80e01-bf85-4b0e-bf3f-7282c5c2d377" (UID: "bee80e01-bf85-4b0e-bf3f-7282c5c2d377"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:03:26.246691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.246658    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:26.246691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.246684    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:26.246691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.246694    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:26.246958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.246701    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:26.246958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.246710    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9865p\" (UniqueName: \"kubernetes.io/projected/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-kube-api-access-9865p\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:26.246958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.246720    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:26.246958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.246727    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bee80e01-bf85-4b0e-bf3f-7282c5c2d377-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:03:26.364487 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.364449    2569 generic.go:356] "Generic (PLEG): container finished" podID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerID="f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79" exitCode=0
Sep 05 08:03:26.364709 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.364511    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"
Sep 05 08:03:26.364709 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.364570    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" event={"ID":"bee80e01-bf85-4b0e-bf3f-7282c5c2d377","Type":"ContainerDied","Data":"f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79"}
Sep 05 08:03:26.364709 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.364647    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp" event={"ID":"bee80e01-bf85-4b0e-bf3f-7282c5c2d377","Type":"ContainerDied","Data":"10379b5cc4a5c017c77c2cd8f8adfce3a4532b09c33860118d570b90b5dd8f91"}
Sep 05 08:03:26.364709 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.364670    2569 scope.go:117] "RemoveContainer" containerID="f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79"
Sep 05 08:03:26.375064 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.375046    2569 scope.go:117] "RemoveContainer" containerID="f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553"
Sep 05 08:03:26.384494 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.384476    2569 scope.go:117] "RemoveContainer" containerID="f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79"
Sep 05 08:03:26.384795 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:03:26.384777    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79\": container with ID starting with f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79 not found: ID does not exist" containerID="f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79"
Sep 05 08:03:26.384841 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.384804    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79"} err="failed to get container status \"f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79\": rpc error: code = NotFound desc = could not find container \"f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79\": container with ID starting with f0ff701cbd36fc75905c41f29c2eeea4ac65e3bfe3a3f30098313d2fea849b79 not found: ID does not exist"
Sep 05 08:03:26.384841 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.384822    2569 scope.go:117] "RemoveContainer" containerID="f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553"
Sep 05 08:03:26.385073 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:03:26.385052    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553\": container with ID starting with f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553 not found: ID does not exist" containerID="f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553"
Sep 05 08:03:26.385136 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.385083    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553"} err="failed to get container status \"f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553\": rpc error: code = NotFound desc = could not find container \"f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553\": container with ID starting with f5ae7dfed159aaf8d558da9bd24d401d7ff39e027143e49d1d789de3ab455553 not found: ID does not exist"
Sep 05 08:03:26.392011 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.391983    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"]
Sep 05 08:03:26.395508 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:26.395484    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-46865726-openshift-default-5d65d49f68-s5ztp"]
Sep 05 08:03:27.571194 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:27.571162    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" path="/var/lib/kubelet/pods/bee80e01-bf85-4b0e-bf3f-7282c5c2d377/volumes"
Sep 05 08:03:39.567902 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:39.567813    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:03:39.571992 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:03:39.571971    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:04:00.422706 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.422665    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"]
Sep 05 08:04:00.423340 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423318    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1ed0ff2-2dfc-4dc0-af8d-549c299bc293" containerName="maas-api"
Sep 05 08:04:00.423432 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423343    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ed0ff2-2dfc-4dc0-af8d-549c299bc293" containerName="maas-api"
Sep 05 08:04:00.423432 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423369    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy"
Sep 05 08:04:00.423432 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423378    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy"
Sep 05 08:04:00.423432 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423399    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy"
Sep 05 08:04:00.423432 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423408    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy"
Sep 05 08:04:00.423432 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423419    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8595634f-0619-4911-a7cf-2328f4774a9f" containerName="maas-api"
Sep 05 08:04:00.423432 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423427    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8595634f-0619-4911-a7cf-2328f4774a9f" containerName="maas-api"
Sep 05 08:04:00.423821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423439    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy"
Sep 05 08:04:00.423821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423447    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy"
Sep 05 08:04:00.423821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423478    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy"
Sep 05 08:04:00.423821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423486    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy"
Sep 05 08:04:00.423821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423614    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy"
Sep 05 08:04:00.423821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423630    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1ed0ff2-2dfc-4dc0-af8d-549c299bc293" containerName="maas-api"
Sep 05 08:04:00.423821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423639    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy"
Sep 05 08:04:00.423821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423648    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc2a913f-bebd-468b-ad62-c3d297fe9ebd" containerName="istio-proxy"
Sep 05 08:04:00.423821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423659    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8595634f-0619-4911-a7cf-2328f4774a9f" containerName="maas-api"
Sep 05 08:04:00.423821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.423669    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bee80e01-bf85-4b0e-bf3f-7282c5c2d377" containerName="istio-proxy"
Sep 05 08:04:00.428362 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.428342    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.434654 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.434631    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-9cecc80d-kserve-self-signed-certs\""
Sep 05 08:04:00.437477 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.437452    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"]
Sep 05 08:04:00.490058 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.490021    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-dshm\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.490230 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.490065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2v48\" (UniqueName: \"kubernetes.io/projected/ad102eb2-5431-4cd9-b178-5107e6bcd198-kube-api-access-v2v48\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.490230 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.490130    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-home\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.490230 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.490210    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-tmp-dir\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.490373 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.490261    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad102eb2-5431-4cd9-b178-5107e6bcd198-tls-certs\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.490373 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.490293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-model-cache\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.590881 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.590846    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-dshm\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.591085 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.590893    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2v48\" (UniqueName: \"kubernetes.io/projected/ad102eb2-5431-4cd9-b178-5107e6bcd198-kube-api-access-v2v48\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.591085 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.590919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-home\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.591085 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.590961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-tmp-dir\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.591085 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.591003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad102eb2-5431-4cd9-b178-5107e6bcd198-tls-certs\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.591085 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.591037    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-model-cache\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.591400 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.591376    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-tmp-dir\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.591459 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.591407    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-home\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.591459 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.591429    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-model-cache\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.593308 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.593285    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-dshm\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.593730 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.593710    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad102eb2-5431-4cd9-b178-5107e6bcd198-tls-certs\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.601699 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.601674    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2v48\" (UniqueName: \"kubernetes.io/projected/ad102eb2-5431-4cd9-b178-5107e6bcd198-kube-api-access-v2v48\") pod \"e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") " pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:00.738422 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:00.738322    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:01.015567 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:01.015520    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"]
Sep 05 08:04:01.017643 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:04:01.017603    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad102eb2_5431_4cd9_b178_5107e6bcd198.slice/crio-bc699f310b4b7356aeb2f72a155a5bd1a2f3dee6aa851a16d9036fb94610b9f5 WatchSource:0}: Error finding container bc699f310b4b7356aeb2f72a155a5bd1a2f3dee6aa851a16d9036fb94610b9f5: Status 404 returned error can't find the container with id bc699f310b4b7356aeb2f72a155a5bd1a2f3dee6aa851a16d9036fb94610b9f5
Sep 05 08:04:01.557297 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:01.557260    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk" event={"ID":"ad102eb2-5431-4cd9-b178-5107e6bcd198","Type":"ContainerStarted","Data":"e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4"}
Sep 05 08:04:01.557297 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:01.557302    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk" event={"ID":"ad102eb2-5431-4cd9-b178-5107e6bcd198","Type":"ContainerStarted","Data":"bc699f310b4b7356aeb2f72a155a5bd1a2f3dee6aa851a16d9036fb94610b9f5"}
Sep 05 08:04:01.583186 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:01.583138    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk" podStartSLOduration=1.583121819 podStartE2EDuration="1.583121819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:04:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:04:01.581581041 +0000 UTC m=+922.526902842" watchObservedRunningTime="2026-09-05 08:04:01.583121819 +0000 UTC m=+922.528443656"
Sep 05 08:04:10.739292 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.739245    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:10.739790 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.739309    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:10.753335 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.753303    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:10.950679 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.950645    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"]
Sep 05 08:04:10.954799 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.954780    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:10.959040 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.959015    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-48a4b5-openshift-default-dockercfg-462sr\""
Sep 05 08:04:10.968603 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.968578    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"]
Sep 05 08:04:10.999176 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.999093    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-data\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:10.999176 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.999148    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-certs\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:10.999420 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.999245    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfmw4\" (UniqueName: \"kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-kube-api-access-tfmw4\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:10.999420 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.999303    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64221411-fade-4538-a5db-b9916919dc2c-istiod-ca-cert\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:10.999420 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.999343    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-credential-socket\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:10.999611 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.999424    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-envoy\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:10.999611 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.999461    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-socket\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:10.999611 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.999489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-istio-token\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:10.999611 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:10.999547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64221411-fade-4538-a5db-b9916919dc2c-istio-podinfo\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.100445 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.100407    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64221411-fade-4538-a5db-b9916919dc2c-istiod-ca-cert\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.100445 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.100452    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-credential-socket\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.100756 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.100487    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-envoy\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.100756 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.100519    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-socket\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.100756 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.100574    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-istio-token\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.100756 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.100610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64221411-fade-4538-a5db-b9916919dc2c-istio-podinfo\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.100756 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.100649    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-data\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.100756 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.100689    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-certs\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.100756 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.100738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfmw4\" (UniqueName: \"kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-kube-api-access-tfmw4\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.101119 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.100998    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-credential-socket\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.101119 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.101068    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-socket\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.101237 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.101172    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-certs\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.101320 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.101299    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64221411-fade-4538-a5db-b9916919dc2c-istiod-ca-cert\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.101380 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.101345    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-data\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.102970 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.102949    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-envoy\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.103186 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.103166    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64221411-fade-4538-a5db-b9916919dc2c-istio-podinfo\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.109619 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.109594    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-istio-token\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.109718 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.109703    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfmw4\" (UniqueName: \"kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-kube-api-access-tfmw4\") pod \"e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") " pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.268655 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.268564    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:11.548297 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.548208    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"]
Sep 05 08:04:11.551441 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:04:11.551416    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64221411_fade_4538_a5db_b9916919dc2c.slice/crio-078952bbbfad3897cc2ddec5b8b3c4a5f291b714f14ba42bfb9a52e9f9e99826 WatchSource:0}: Error finding container 078952bbbfad3897cc2ddec5b8b3c4a5f291b714f14ba42bfb9a52e9f9e99826: Status 404 returned error can't find the container with id 078952bbbfad3897cc2ddec5b8b3c4a5f291b714f14ba42bfb9a52e9f9e99826
Sep 05 08:04:11.556719 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.556682    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:04:11.556833 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.556758    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:04:11.556833 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.556795    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:04:11.611170 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.611143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" event={"ID":"64221411-fade-4538-a5db-b9916919dc2c","Type":"ContainerStarted","Data":"078952bbbfad3897cc2ddec5b8b3c4a5f291b714f14ba42bfb9a52e9f9e99826"}
Sep 05 08:04:11.623687 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:11.623664    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:12.618974 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:12.618940    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" event={"ID":"64221411-fade-4538-a5db-b9916919dc2c","Type":"ContainerStarted","Data":"9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748"}
Sep 05 08:04:12.649736 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:12.649683    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" podStartSLOduration=2.6496655479999998 podStartE2EDuration="2.649665548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:04:12.646889909 +0000 UTC m=+933.592211725" watchObservedRunningTime="2026-09-05 08:04:12.649665548 +0000 UTC m=+933.594987352"
Sep 05 08:04:13.269608 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:13.269567    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:13.274846 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:13.274821    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:13.623907 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:13.623878    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:13.624965 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:13.624945    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:15.816203 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:15.816167    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"]
Sep 05 08:04:15.820334 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:15.820314    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:15.827847 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:15.827828    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c62fz\""
Sep 05 08:04:15.829023 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:15.829007    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-48a4b5-serving-cert\""
Sep 05 08:04:15.851175 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:15.851149    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"]
Sep 05 08:04:15.950878 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:15.950827    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-maas-api-tls\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:15.951059 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:15.950914    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-metrics-certs\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:15.951059 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:15.950969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvhql\" (UniqueName: \"kubernetes.io/projected/10b9de43-193c-4fd3-bafb-eb9ee9c59282-kube-api-access-qvhql\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:15.951059 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:15.951016    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10b9de43-193c-4fd3-bafb-eb9ee9c59282-openshift-service-ca\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:16.051804 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:16.051774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10b9de43-193c-4fd3-bafb-eb9ee9c59282-openshift-service-ca\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:16.051939 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:16.051825    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-maas-api-tls\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:16.051939 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:16.051867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-metrics-certs\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:16.051939 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:16.051902    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvhql\" (UniqueName: \"kubernetes.io/projected/10b9de43-193c-4fd3-bafb-eb9ee9c59282-kube-api-access-qvhql\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:16.052079 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:04:16.052012    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-serving-cert: secret "maas-api-e2e-explicit-48a4b5-serving-cert" not found
Sep 05 08:04:16.052079 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:04:16.052076    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-maas-api-tls podName:10b9de43-193c-4fd3-bafb-eb9ee9c59282 nodeName:}" failed. No retries permitted until 2026-09-05 08:04:16.552056667 +0000 UTC m=+937.497378449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-maas-api-tls") pod "maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p" (UID: "10b9de43-193c-4fd3-bafb-eb9ee9c59282") : secret "maas-api-e2e-explicit-48a4b5-serving-cert" not found
Sep 05 08:04:16.052504 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:16.052470    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10b9de43-193c-4fd3-bafb-eb9ee9c59282-openshift-service-ca\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:16.054544 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:16.054507    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-metrics-certs\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:16.061170 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:16.061143    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvhql\" (UniqueName: \"kubernetes.io/projected/10b9de43-193c-4fd3-bafb-eb9ee9c59282-kube-api-access-qvhql\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:16.556914 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:16.556874    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-maas-api-tls\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:16.559451 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:16.559430    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-maas-api-tls\") pod \"maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:16.731761 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:16.731731    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:17.019186 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:17.019157    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"]
Sep 05 08:04:17.020700 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:04:17.020675    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b9de43_193c_4fd3_bafb_eb9ee9c59282.slice/crio-f6e338aa3f077b3da08324e3cb353cc6185f7342419674d8e7aeb41ab3bdb32a WatchSource:0}: Error finding container f6e338aa3f077b3da08324e3cb353cc6185f7342419674d8e7aeb41ab3bdb32a: Status 404 returned error can't find the container with id f6e338aa3f077b3da08324e3cb353cc6185f7342419674d8e7aeb41ab3bdb32a
Sep 05 08:04:17.649258 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:17.649167    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p" event={"ID":"10b9de43-193c-4fd3-bafb-eb9ee9c59282","Type":"ContainerStarted","Data":"18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf"}
Sep 05 08:04:17.649258 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:17.649228    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p" event={"ID":"10b9de43-193c-4fd3-bafb-eb9ee9c59282","Type":"ContainerStarted","Data":"f6e338aa3f077b3da08324e3cb353cc6185f7342419674d8e7aeb41ab3bdb32a"}
Sep 05 08:04:17.649448 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:17.649260    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:17.680634 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:17.680581    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p" podStartSLOduration=2.319414473 podStartE2EDuration="2.680566479s" totalImagesPullingTime="361.152006ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:04:17.669785466 +0000 UTC m=+938.615107272" watchObservedRunningTime="2026-09-05 08:04:17.680566479 +0000 UTC m=+938.625888284"
Sep 05 08:04:22.778064 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:22.778030    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"]
Sep 05 08:04:23.659792 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:23.659763    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:04:36.761622 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:36.761583    2569 generic.go:356] "Generic (PLEG): container finished" podID="64221411-fade-4538-a5db-b9916919dc2c" containerID="9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748" exitCode=0
Sep 05 08:04:36.762061 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:36.761665    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" event={"ID":"64221411-fade-4538-a5db-b9916919dc2c","Type":"ContainerDied","Data":"9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748"}
Sep 05 08:04:36.762219 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:36.762202    2569 scope.go:117] "RemoveContainer" containerID="9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748"
Sep 05 08:04:36.763544 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:36.763498    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:04:36.763630 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:36.763600    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:04:36.763671 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:36.763634    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:04:37.771148 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:37.771120    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" event={"ID":"64221411-fade-4538-a5db-b9916919dc2c","Type":"ContainerStarted","Data":"619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448"}
Sep 05 08:04:37.800551 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:37.800499    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk" podUID="ad102eb2-5431-4cd9-b178-5107e6bcd198" containerName="main" containerID="cri-o://e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4" gracePeriod=45
Sep 05 08:04:38.100202 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.100181    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:38.178280 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.178248    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-model-cache\") pod \"ad102eb2-5431-4cd9-b178-5107e6bcd198\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") "
Sep 05 08:04:38.178595 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.178539    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad102eb2-5431-4cd9-b178-5107e6bcd198-tls-certs\") pod \"ad102eb2-5431-4cd9-b178-5107e6bcd198\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") "
Sep 05 08:04:38.178595 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.178569    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad102eb2-5431-4cd9-b178-5107e6bcd198" (UID: "ad102eb2-5431-4cd9-b178-5107e6bcd198"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:38.178804 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.178753    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-home\") pod \"ad102eb2-5431-4cd9-b178-5107e6bcd198\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") "
Sep 05 08:04:38.178804 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.178775    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-tmp-dir\") pod \"ad102eb2-5431-4cd9-b178-5107e6bcd198\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") "
Sep 05 08:04:38.178981 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.178942    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-home" (OuterVolumeSpecName: "home") pod "ad102eb2-5431-4cd9-b178-5107e6bcd198" (UID: "ad102eb2-5431-4cd9-b178-5107e6bcd198"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:38.179061 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.178989    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2v48\" (UniqueName: \"kubernetes.io/projected/ad102eb2-5431-4cd9-b178-5107e6bcd198-kube-api-access-v2v48\") pod \"ad102eb2-5431-4cd9-b178-5107e6bcd198\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") "
Sep 05 08:04:38.179116 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.179058    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad102eb2-5431-4cd9-b178-5107e6bcd198" (UID: "ad102eb2-5431-4cd9-b178-5107e6bcd198"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:38.179116 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.179073    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-dshm\") pod \"ad102eb2-5431-4cd9-b178-5107e6bcd198\" (UID: \"ad102eb2-5431-4cd9-b178-5107e6bcd198\") "
Sep 05 08:04:38.179930 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.179863    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-model-cache\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:04:38.180092 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.180079    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-home\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:04:38.180217 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.180204    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-tmp-dir\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:04:38.181045 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.181018    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad102eb2-5431-4cd9-b178-5107e6bcd198-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad102eb2-5431-4cd9-b178-5107e6bcd198" (UID: "ad102eb2-5431-4cd9-b178-5107e6bcd198"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:04:38.181197 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.181162    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-dshm" (OuterVolumeSpecName: "dshm") pod "ad102eb2-5431-4cd9-b178-5107e6bcd198" (UID: "ad102eb2-5431-4cd9-b178-5107e6bcd198"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:04:38.181728 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.181707    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad102eb2-5431-4cd9-b178-5107e6bcd198-kube-api-access-v2v48" (OuterVolumeSpecName: "kube-api-access-v2v48") pod "ad102eb2-5431-4cd9-b178-5107e6bcd198" (UID: "ad102eb2-5431-4cd9-b178-5107e6bcd198"). InnerVolumeSpecName "kube-api-access-v2v48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:04:38.269607 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.269573    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:38.269876 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.269853    2569 patch_prober.go:28] interesting pod/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 05 08:04:38.269930 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.269902    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 05 08:04:38.280905 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.280875    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2v48\" (UniqueName: \"kubernetes.io/projected/ad102eb2-5431-4cd9-b178-5107e6bcd198-kube-api-access-v2v48\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:04:38.280905 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.280900    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad102eb2-5431-4cd9-b178-5107e6bcd198-dshm\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:04:38.280905 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.280909    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad102eb2-5431-4cd9-b178-5107e6bcd198-tls-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:04:38.777142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.777110    2569 generic.go:356] "Generic (PLEG): container finished" podID="ad102eb2-5431-4cd9-b178-5107e6bcd198" containerID="e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4" exitCode=0
Sep 05 08:04:38.777627 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.777222    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"
Sep 05 08:04:38.777627 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.777244    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk" event={"ID":"ad102eb2-5431-4cd9-b178-5107e6bcd198","Type":"ContainerDied","Data":"e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4"}
Sep 05 08:04:38.777627 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.777280    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk" event={"ID":"ad102eb2-5431-4cd9-b178-5107e6bcd198","Type":"ContainerDied","Data":"bc699f310b4b7356aeb2f72a155a5bd1a2f3dee6aa851a16d9036fb94610b9f5"}
Sep 05 08:04:38.777627 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.777297    2569 scope.go:117] "RemoveContainer" containerID="e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4"
Sep 05 08:04:38.787382 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.787364    2569 scope.go:117] "RemoveContainer" containerID="e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4"
Sep 05 08:04:38.787678 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:04:38.787656    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4\": container with ID starting with e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4 not found: ID does not exist" containerID="e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4"
Sep 05 08:04:38.787782 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.787685    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4"} err="failed to get container status \"e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4\": rpc error: code = NotFound desc = could not find container \"e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4\": container with ID starting with e4177f7749d1aed2f876a32ae90e6d5b34a9f9afc5a54db2ffdcdf3fd4ca06b4 not found: ID does not exist"
Sep 05 08:04:38.800191 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.800166    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"]
Sep 05 08:04:38.807368 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:38.807348    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-9cecc80d-kserve-8b49c4fd-x56wk"]
Sep 05 08:04:39.269279 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:39.269243    2569 patch_prober.go:28] interesting pod/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 05 08:04:39.269475 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:39.269313    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 05 08:04:39.572467 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:39.572439    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad102eb2-5431-4cd9-b178-5107e6bcd198" path="/var/lib/kubelet/pods/ad102eb2-5431-4cd9-b178-5107e6bcd198/volumes"
Sep 05 08:04:40.269411 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:40.269370    2569 patch_prober.go:28] interesting pod/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 05 08:04:40.269858 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:40.269437    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 05 08:04:41.269549 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:41.269487    2569 patch_prober.go:28] interesting pod/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 05 08:04:41.269943 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:41.269584    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 05 08:04:42.269416 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:42.269374    2569 patch_prober.go:28] interesting pod/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 05 08:04:42.269622 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:42.269450    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 05 08:04:43.272993 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:43.272964    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:43.273466 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:43.273342    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:43.274071 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:43.274053    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:04:58.445843 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.445811    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"]
Sep 05 08:04:58.446305 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.446291    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad102eb2-5431-4cd9-b178-5107e6bcd198" containerName="main"
Sep 05 08:04:58.446350 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.446313    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad102eb2-5431-4cd9-b178-5107e6bcd198" containerName="main"
Sep 05 08:04:58.446451 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.446441    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad102eb2-5431-4cd9-b178-5107e6bcd198" containerName="main"
Sep 05 08:04:58.450155 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.450129    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.455753 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.455723    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-66998bbc-openshift-default-dockercfg-xzbxr\""
Sep 05 08:04:58.465584 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.465517    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"]
Sep 05 08:04:58.587942 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.587904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb7cc9d9-598f-4370-ad98-fc7710f74791-istiod-ca-cert\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.587942 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.587947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-token\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.588156 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.587969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-envoy\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.588156 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.588084    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-credential-socket\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.588156 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.588123    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fk9b\" (UniqueName: \"kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-kube-api-access-6fk9b\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.588295 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.588170    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-socket\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.588295 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.588213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-podinfo\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.588368 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.588298    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-certs\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.588368 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.588316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-data\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.689589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.689554    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-token\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.689762 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.689610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-envoy\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.689762 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.689673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-credential-socket\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.689762 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.689707    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fk9b\" (UniqueName: \"kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-kube-api-access-6fk9b\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.689762 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.689748    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-socket\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.690061 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.689816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-podinfo\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.690061 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.689897    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-certs\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.690061 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.689924    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-data\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.690061 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.689957    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb7cc9d9-598f-4370-ad98-fc7710f74791-istiod-ca-cert\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.690272 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.690141    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-credential-socket\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.690412 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.690388    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-certs\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.690566 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.690410    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-socket\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.690566 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.690555    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-data\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.690837 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.690819    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb7cc9d9-598f-4370-ad98-fc7710f74791-istiod-ca-cert\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.692520 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.692491    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-envoy\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.692890 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.692869    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-podinfo\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.700404 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.700348    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-token\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.701041 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.701014    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fk9b\" (UniqueName: \"kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-kube-api-access-6fk9b\") pod \"e2e-mt-66998bbc-openshift-default-6569645764-gczhs\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") " pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:58.766915 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:58.766885    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:04:59.053743 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:59.053711    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"]
Sep 05 08:04:59.055745 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:04:59.055710    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7cc9d9_598f_4370_ad98_fc7710f74791.slice/crio-50dcb0589d2304ec68a428fca027c454e9599add4c3e19c66caf46e61b213ea5 WatchSource:0}: Error finding container 50dcb0589d2304ec68a428fca027c454e9599add4c3e19c66caf46e61b213ea5: Status 404 returned error can't find the container with id 50dcb0589d2304ec68a428fca027c454e9599add4c3e19c66caf46e61b213ea5
Sep 05 08:04:59.058156 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:59.058126    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:04:59.058257 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:59.058190    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:04:59.058257 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:59.058215    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:04:59.898274 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:59.898241    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" event={"ID":"eb7cc9d9-598f-4370-ad98-fc7710f74791","Type":"ContainerStarted","Data":"b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a"}
Sep 05 08:04:59.898672 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:59.898315    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" event={"ID":"eb7cc9d9-598f-4370-ad98-fc7710f74791","Type":"ContainerStarted","Data":"50dcb0589d2304ec68a428fca027c454e9599add4c3e19c66caf46e61b213ea5"}
Sep 05 08:04:59.922093 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:04:59.922050    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" podStartSLOduration=1.922035464 podStartE2EDuration="1.922035464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:04:59.920634506 +0000 UTC m=+980.865956335" watchObservedRunningTime="2026-09-05 08:04:59.922035464 +0000 UTC m=+980.867357268"
Sep 05 08:05:00.768030 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:00.767958    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:05:00.773129 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:00.773106    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:05:00.903439 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:00.903416    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:05:00.904362 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:00.904344    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:05:03.053760 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.053718    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"]
Sep 05 08:05:03.056334 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.056315    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.059158 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.059140    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-66998bbc-serving-cert\""
Sep 05 08:05:03.087930 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.087909    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"]
Sep 05 08:05:03.239753 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.239719    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d89mq\" (UniqueName: \"kubernetes.io/projected/b221b02b-3291-4508-aa51-e60aa7533729-kube-api-access-d89mq\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.239925 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.239805    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-maas-api-tls\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.239925 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.239885    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-metrics-certs\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.240004 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.239924    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b221b02b-3291-4508-aa51-e60aa7533729-openshift-service-ca\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.340687 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.340650    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b221b02b-3291-4508-aa51-e60aa7533729-openshift-service-ca\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.340871 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.340728    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d89mq\" (UniqueName: \"kubernetes.io/projected/b221b02b-3291-4508-aa51-e60aa7533729-kube-api-access-d89mq\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.340871 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.340775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-maas-api-tls\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.340871 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.340846    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-metrics-certs\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.341642 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.341614    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b221b02b-3291-4508-aa51-e60aa7533729-openshift-service-ca\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.343312 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.343286    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-maas-api-tls\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.343400 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.343338    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-metrics-certs\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.349864 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.349845    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d89mq\" (UniqueName: \"kubernetes.io/projected/b221b02b-3291-4508-aa51-e60aa7533729-kube-api-access-d89mq\") pod \"maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.367412 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.367392    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:03.654241 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.654213    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"]
Sep 05 08:05:03.657616 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:05:03.657579    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb221b02b_3291_4508_aa51_e60aa7533729.slice/crio-19dbbc0decca76a5cf4350e1e446977b622dabd0a395459ec7ce4e373cff2dd5 WatchSource:0}: Error finding container 19dbbc0decca76a5cf4350e1e446977b622dabd0a395459ec7ce4e373cff2dd5: Status 404 returned error can't find the container with id 19dbbc0decca76a5cf4350e1e446977b622dabd0a395459ec7ce4e373cff2dd5
Sep 05 08:05:03.920356 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:03.920276    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2" event={"ID":"b221b02b-3291-4508-aa51-e60aa7533729","Type":"ContainerStarted","Data":"19dbbc0decca76a5cf4350e1e446977b622dabd0a395459ec7ce4e373cff2dd5"}
Sep 05 08:05:04.927403 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:04.927365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2" event={"ID":"b221b02b-3291-4508-aa51-e60aa7533729","Type":"ContainerStarted","Data":"0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa"}
Sep 05 08:05:04.927830 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:04.927426    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:04.949979 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:04.949928    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2" podStartSLOduration=1.417202837 podStartE2EDuration="1.949912236s" totalImagesPullingTime="532.709399ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:05:04.948135218 +0000 UTC m=+985.893457023" watchObservedRunningTime="2026-09-05 08:05:04.949912236 +0000 UTC m=+985.895234064"
Sep 05 08:05:10.938805 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:10.938774    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:29.333729 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.333694    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"]
Sep 05 08:05:29.334232 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.334009    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p" podUID="10b9de43-193c-4fd3-bafb-eb9ee9c59282" containerName="maas-api" containerID="cri-o://18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf" gracePeriod=30
Sep 05 08:05:29.656560 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.652257    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:05:29.802030 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.802000    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-metrics-certs\") pod \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") "
Sep 05 08:05:29.802206 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.802047    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10b9de43-193c-4fd3-bafb-eb9ee9c59282-openshift-service-ca\") pod \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") "
Sep 05 08:05:29.802206 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.802177    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-maas-api-tls\") pod \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") "
Sep 05 08:05:29.802206 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.802199    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvhql\" (UniqueName: \"kubernetes.io/projected/10b9de43-193c-4fd3-bafb-eb9ee9c59282-kube-api-access-qvhql\") pod \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\" (UID: \"10b9de43-193c-4fd3-bafb-eb9ee9c59282\") "
Sep 05 08:05:29.802393 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.802370    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10b9de43-193c-4fd3-bafb-eb9ee9c59282-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10b9de43-193c-4fd3-bafb-eb9ee9c59282" (UID: "10b9de43-193c-4fd3-bafb-eb9ee9c59282"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:05:29.802602 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.802581    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10b9de43-193c-4fd3-bafb-eb9ee9c59282-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:29.804393 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.804361    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10b9de43-193c-4fd3-bafb-eb9ee9c59282" (UID: "10b9de43-193c-4fd3-bafb-eb9ee9c59282"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:05:29.804499 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.804407    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "10b9de43-193c-4fd3-bafb-eb9ee9c59282" (UID: "10b9de43-193c-4fd3-bafb-eb9ee9c59282"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:05:29.804499 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.804429    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10b9de43-193c-4fd3-bafb-eb9ee9c59282-kube-api-access-qvhql" (OuterVolumeSpecName: "kube-api-access-qvhql") pod "10b9de43-193c-4fd3-bafb-eb9ee9c59282" (UID: "10b9de43-193c-4fd3-bafb-eb9ee9c59282"). InnerVolumeSpecName "kube-api-access-qvhql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:05:29.903552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.903461    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:29.903552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.903489    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvhql\" (UniqueName: \"kubernetes.io/projected/10b9de43-193c-4fd3-bafb-eb9ee9c59282-kube-api-access-qvhql\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:29.903552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:29.903498    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10b9de43-193c-4fd3-bafb-eb9ee9c59282-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:30.085780 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:30.085743    2569 generic.go:356] "Generic (PLEG): container finished" podID="10b9de43-193c-4fd3-bafb-eb9ee9c59282" containerID="18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf" exitCode=0
Sep 05 08:05:30.085948 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:30.085788    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"
Sep 05 08:05:30.085948 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:30.085835    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p" event={"ID":"10b9de43-193c-4fd3-bafb-eb9ee9c59282","Type":"ContainerDied","Data":"18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf"}
Sep 05 08:05:30.085948 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:30.085883    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p" event={"ID":"10b9de43-193c-4fd3-bafb-eb9ee9c59282","Type":"ContainerDied","Data":"f6e338aa3f077b3da08324e3cb353cc6185f7342419674d8e7aeb41ab3bdb32a"}
Sep 05 08:05:30.085948 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:30.085910    2569 scope.go:117] "RemoveContainer" containerID="18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf"
Sep 05 08:05:30.095830 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:30.095806    2569 scope.go:117] "RemoveContainer" containerID="18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf"
Sep 05 08:05:30.096104 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:05:30.096083    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf\": container with ID starting with 18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf not found: ID does not exist" containerID="18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf"
Sep 05 08:05:30.096175 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:30.096112    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf"} err="failed to get container status \"18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf\": rpc error: code = NotFound desc = could not find container \"18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf\": container with ID starting with 18015c3bdc816d8743de4372c6209e95ea7b9f58c7cf5c59a0d72a412911d3bf not found: ID does not exist"
Sep 05 08:05:30.110093 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:30.110073    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"]
Sep 05 08:05:30.114822 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:30.114794    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-48a4b5-7b6594db99-ntn6p"]
Sep 05 08:05:31.571552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:31.571481    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10b9de43-193c-4fd3-bafb-eb9ee9c59282" path="/var/lib/kubelet/pods/10b9de43-193c-4fd3-bafb-eb9ee9c59282/volumes"
Sep 05 08:05:31.944719 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:31.944614    2569 patch_prober.go:28] interesting pod/e2e-mt-66998bbc-openshift-default-6569645764-gczhs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 08:05:31.944719 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:31.944674    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 08:05:34.113290 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:34.113260    2569 generic.go:356] "Generic (PLEG): container finished" podID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerID="b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a" exitCode=0
Sep 05 08:05:34.113707 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:34.113331    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" event={"ID":"eb7cc9d9-598f-4370-ad98-fc7710f74791","Type":"ContainerDied","Data":"b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a"}
Sep 05 08:05:34.113773 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:34.113759    2569 scope.go:117] "RemoveContainer" containerID="b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a"
Sep 05 08:05:34.114869 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:34.114844    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:05:34.114952 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:34.114910    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:05:34.114952 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:34.114948    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:05:35.121088 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:35.121051    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" event={"ID":"eb7cc9d9-598f-4370-ad98-fc7710f74791","Type":"ContainerStarted","Data":"b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8"}
Sep 05 08:05:35.767409 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:35.767370    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:05:35.767674 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:35.767651    2569 patch_prober.go:28] interesting pod/e2e-mt-66998bbc-openshift-default-6569645764-gczhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 08:05:35.767745 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:35.767711    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 08:05:36.776773 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:36.776741    2569 patch_prober.go:28] interesting pod/e2e-mt-66998bbc-openshift-default-6569645764-gczhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 08:05:36.777316 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:36.777286    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 08:05:36.782016 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:36.781973    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"]
Sep 05 08:05:36.782414 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:36.782385    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy" containerID="cri-o://619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448" gracePeriod=30
Sep 05 08:05:37.556915 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.556884    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"]
Sep 05 08:05:37.557919 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.557894    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10b9de43-193c-4fd3-bafb-eb9ee9c59282" containerName="maas-api"
Sep 05 08:05:37.562552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.558280    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="10b9de43-193c-4fd3-bafb-eb9ee9c59282" containerName="maas-api"
Sep 05 08:05:37.562929 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.562906    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="10b9de43-193c-4fd3-bafb-eb9ee9c59282" containerName="maas-api"
Sep 05 08:05:37.567798 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.567773    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.571199 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.571178    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-ecd628-openshift-default-dockercfg-zd8v5\""
Sep 05 08:05:37.578038 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.578015    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"]
Sep 05 08:05:37.585743 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.585723    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-socket\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.585849 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.585753    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-token\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.585849 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.585774    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istiod-ca-cert\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.585953 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.585842    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-data\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.585953 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.585916    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-podinfo\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.586035 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.585952    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-certs\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.586090 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.586044    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7xpt\" (UniqueName: \"kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-kube-api-access-x7xpt\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.586144 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.586111    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-envoy\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.586220 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.586203    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-credential-socket\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687028 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.686990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7xpt\" (UniqueName: \"kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-kube-api-access-x7xpt\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687228 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687039    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-envoy\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687228 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687091    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-credential-socket\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687228 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687120    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-socket\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687228 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-token\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687228 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687196    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istiod-ca-cert\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687494 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-data\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687494 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687290    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-podinfo\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687494 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-certs\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687690 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687602    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-socket\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.687766 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687735    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-data\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.688016 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687970    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-certs\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.688016 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.687995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istiod-ca-cert\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.688127 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.688034    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-credential-socket\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.689576 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.689554    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-envoy\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.689976 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.689958    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-podinfo\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.696345 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.696322    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7xpt\" (UniqueName: \"kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-kube-api-access-x7xpt\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.696516 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.696497    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-token\") pod \"e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") " pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:37.767652 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.767612    2569 patch_prober.go:28] interesting pod/e2e-mt-66998bbc-openshift-default-6569645764-gczhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 08:05:37.767797 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.767677    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 08:05:37.881695 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:37.881657    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:38.166319 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:38.166294    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"]
Sep 05 08:05:38.168791 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:05:38.168755    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d50c1b9_572b_4bc0_8bbc_fbe7e460865c.slice/crio-62cb9939b39ace3632e6d7bf569300d6bf28074020b3b4388417fe78e9693169 WatchSource:0}: Error finding container 62cb9939b39ace3632e6d7bf569300d6bf28074020b3b4388417fe78e9693169: Status 404 returned error can't find the container with id 62cb9939b39ace3632e6d7bf569300d6bf28074020b3b4388417fe78e9693169
Sep 05 08:05:38.171200 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:38.171170    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:05:38.171281 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:38.171232    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:05:38.171281 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:38.171261    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:05:38.767740 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:38.767710    2569 patch_prober.go:28] interesting pod/e2e-mt-66998bbc-openshift-default-6569645764-gczhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 08:05:38.767917 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:38.767768    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 08:05:39.151133 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:39.151084    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf" event={"ID":"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c","Type":"ContainerStarted","Data":"eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c"}
Sep 05 08:05:39.151652 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:39.151196    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf" event={"ID":"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c","Type":"ContainerStarted","Data":"62cb9939b39ace3632e6d7bf569300d6bf28074020b3b4388417fe78e9693169"}
Sep 05 08:05:39.178691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:39.178642    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf" podStartSLOduration=2.178625765 podStartE2EDuration="2.178625765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:05:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:05:39.175649191 +0000 UTC m=+1020.120970998" watchObservedRunningTime="2026-09-05 08:05:39.178625765 +0000 UTC m=+1020.123947639"
Sep 05 08:05:39.768142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:39.768107    2569 patch_prober.go:28] interesting pod/e2e-mt-66998bbc-openshift-default-6569645764-gczhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 08:05:39.768335 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:39.768163    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 08:05:39.882150 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:39.882120    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:39.888004 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:39.887982    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:40.159970 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:40.159932    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:40.164042 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:40.164011    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:05:40.767695 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:40.767660    2569 patch_prober.go:28] interesting pod/e2e-mt-66998bbc-openshift-default-6569645764-gczhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 08:05:40.767865 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:40.767718    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 08:05:41.771444 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:41.771368    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:05:41.771810 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:41.771657    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:05:41.772222 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:41.772208    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:05:42.050788 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.050767    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:05:42.130156 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130128    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-certs\") pod \"64221411-fade-4538-a5db-b9916919dc2c\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") "
Sep 05 08:05:42.130332 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130193    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-credential-socket\") pod \"64221411-fade-4538-a5db-b9916919dc2c\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") "
Sep 05 08:05:42.130332 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130245    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfmw4\" (UniqueName: \"kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-kube-api-access-tfmw4\") pod \"64221411-fade-4538-a5db-b9916919dc2c\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") "
Sep 05 08:05:42.130332 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130279    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-istio-token\") pod \"64221411-fade-4538-a5db-b9916919dc2c\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") "
Sep 05 08:05:42.130332 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130302    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-socket\") pod \"64221411-fade-4538-a5db-b9916919dc2c\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") "
Sep 05 08:05:42.130613 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130339    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64221411-fade-4538-a5db-b9916919dc2c-istiod-ca-cert\") pod \"64221411-fade-4538-a5db-b9916919dc2c\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") "
Sep 05 08:05:42.130613 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130383    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-envoy\") pod \"64221411-fade-4538-a5db-b9916919dc2c\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") "
Sep 05 08:05:42.130613 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130410    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-data\") pod \"64221411-fade-4538-a5db-b9916919dc2c\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") "
Sep 05 08:05:42.130613 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130416    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "64221411-fade-4538-a5db-b9916919dc2c" (UID: "64221411-fade-4538-a5db-b9916919dc2c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:42.130613 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130452    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64221411-fade-4538-a5db-b9916919dc2c-istio-podinfo\") pod \"64221411-fade-4538-a5db-b9916919dc2c\" (UID: \"64221411-fade-4538-a5db-b9916919dc2c\") "
Sep 05 08:05:42.130613 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130564    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "64221411-fade-4538-a5db-b9916919dc2c" (UID: "64221411-fade-4538-a5db-b9916919dc2c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:42.130908 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130667    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "64221411-fade-4538-a5db-b9916919dc2c" (UID: "64221411-fade-4538-a5db-b9916919dc2c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:42.130908 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130685    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-data" (OuterVolumeSpecName: "istio-data") pod "64221411-fade-4538-a5db-b9916919dc2c" (UID: "64221411-fade-4538-a5db-b9916919dc2c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:42.131025 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130883    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:42.131025 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130930    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:42.131025 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130947    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:42.131025 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.130960    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:42.131211 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.131034    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64221411-fade-4538-a5db-b9916919dc2c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "64221411-fade-4538-a5db-b9916919dc2c" (UID: "64221411-fade-4538-a5db-b9916919dc2c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:05:42.132954 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.132932    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-istio-token" (OuterVolumeSpecName: "istio-token") pod "64221411-fade-4538-a5db-b9916919dc2c" (UID: "64221411-fade-4538-a5db-b9916919dc2c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:05:42.133054 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.132978    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-kube-api-access-tfmw4" (OuterVolumeSpecName: "kube-api-access-tfmw4") pod "64221411-fade-4538-a5db-b9916919dc2c" (UID: "64221411-fade-4538-a5db-b9916919dc2c"). InnerVolumeSpecName "kube-api-access-tfmw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:05:42.133176 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.133156    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "64221411-fade-4538-a5db-b9916919dc2c" (UID: "64221411-fade-4538-a5db-b9916919dc2c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:42.133300 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.133177    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/64221411-fade-4538-a5db-b9916919dc2c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "64221411-fade-4538-a5db-b9916919dc2c" (UID: "64221411-fade-4538-a5db-b9916919dc2c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:05:42.173110 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.173079    2569 generic.go:356] "Generic (PLEG): container finished" podID="64221411-fade-4538-a5db-b9916919dc2c" containerID="619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448" exitCode=0
Sep 05 08:05:42.173283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.173132    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"
Sep 05 08:05:42.173283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.173164    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" event={"ID":"64221411-fade-4538-a5db-b9916919dc2c","Type":"ContainerDied","Data":"619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448"}
Sep 05 08:05:42.173283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.173221    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc" event={"ID":"64221411-fade-4538-a5db-b9916919dc2c","Type":"ContainerDied","Data":"078952bbbfad3897cc2ddec5b8b3c4a5f291b714f14ba42bfb9a52e9f9e99826"}
Sep 05 08:05:42.173283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.173244    2569 scope.go:117] "RemoveContainer" containerID="619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448"
Sep 05 08:05:42.183238 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.183221    2569 scope.go:117] "RemoveContainer" containerID="9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748"
Sep 05 08:05:42.191400 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.191380    2569 scope.go:117] "RemoveContainer" containerID="619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448"
Sep 05 08:05:42.191742 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:05:42.191723    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448\": container with ID starting with 619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448 not found: ID does not exist" containerID="619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448"
Sep 05 08:05:42.191789 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.191749    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448"} err="failed to get container status \"619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448\": rpc error: code = NotFound desc = could not find container \"619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448\": container with ID starting with 619d305b05f779bbe9c2d09fa48d64c9b2e7ccef07f2446fde274fe3f96d0448 not found: ID does not exist"
Sep 05 08:05:42.191789 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.191765    2569 scope.go:117] "RemoveContainer" containerID="9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748"
Sep 05 08:05:42.191994 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:05:42.191978    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748\": container with ID starting with 9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748 not found: ID does not exist" containerID="9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748"
Sep 05 08:05:42.192034 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.192000    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748"} err="failed to get container status \"9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748\": rpc error: code = NotFound desc = could not find container \"9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748\": container with ID starting with 9bcc200dd68b83d13e61f4cc9af2b7eef05d2f1c09d0958a0628e8634c2e2748 not found: ID does not exist"
Sep 05 08:05:42.196326 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.196307    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"]
Sep 05 08:05:42.201050 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.201030    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-48a4b5-openshift-default-7999b9486d-zp7hc"]
Sep 05 08:05:42.232144 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.232001    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfmw4\" (UniqueName: \"kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-kube-api-access-tfmw4\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:42.232144 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.232031    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64221411-fade-4538-a5db-b9916919dc2c-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:42.232144 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.232047    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64221411-fade-4538-a5db-b9916919dc2c-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:42.232144 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.232061    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64221411-fade-4538-a5db-b9916919dc2c-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:42.232144 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:42.232075    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64221411-fade-4538-a5db-b9916919dc2c-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:43.571351 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:43.571312    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64221411-fade-4538-a5db-b9916919dc2c" path="/var/lib/kubelet/pods/64221411-fade-4538-a5db-b9916919dc2c/volumes"
Sep 05 08:05:44.897163 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:44.897128    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"]
Sep 05 08:05:44.897623 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:44.897396    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2" podUID="b221b02b-3291-4508-aa51-e60aa7533729" containerName="maas-api" containerID="cri-o://0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa" gracePeriod=30
Sep 05 08:05:45.165183 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.165119    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:45.193352 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.193149    2569 generic.go:356] "Generic (PLEG): container finished" podID="b221b02b-3291-4508-aa51-e60aa7533729" containerID="0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa" exitCode=0
Sep 05 08:05:45.193352 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.193197    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2" event={"ID":"b221b02b-3291-4508-aa51-e60aa7533729","Type":"ContainerDied","Data":"0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa"}
Sep 05 08:05:45.193352 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.193210    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"
Sep 05 08:05:45.193352 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.193247    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2" event={"ID":"b221b02b-3291-4508-aa51-e60aa7533729","Type":"ContainerDied","Data":"19dbbc0decca76a5cf4350e1e446977b622dabd0a395459ec7ce4e373cff2dd5"}
Sep 05 08:05:45.193352 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.193271    2569 scope.go:117] "RemoveContainer" containerID="0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa"
Sep 05 08:05:45.203097 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.203075    2569 scope.go:117] "RemoveContainer" containerID="0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa"
Sep 05 08:05:45.203394 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:05:45.203373    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa\": container with ID starting with 0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa not found: ID does not exist" containerID="0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa"
Sep 05 08:05:45.203482 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.203402    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa"} err="failed to get container status \"0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa\": rpc error: code = NotFound desc = could not find container \"0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa\": container with ID starting with 0f4700b948931874f3ca3454a1087bed5ea9a17eb2733d3a81f56a37f696f2fa not found: ID does not exist"
Sep 05 08:05:45.261938 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.261910    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b221b02b-3291-4508-aa51-e60aa7533729-openshift-service-ca\") pod \"b221b02b-3291-4508-aa51-e60aa7533729\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") "
Sep 05 08:05:45.262083 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.261995    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-metrics-certs\") pod \"b221b02b-3291-4508-aa51-e60aa7533729\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") "
Sep 05 08:05:45.262083 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.262039    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d89mq\" (UniqueName: \"kubernetes.io/projected/b221b02b-3291-4508-aa51-e60aa7533729-kube-api-access-d89mq\") pod \"b221b02b-3291-4508-aa51-e60aa7533729\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") "
Sep 05 08:05:45.262083 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.262076    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-maas-api-tls\") pod \"b221b02b-3291-4508-aa51-e60aa7533729\" (UID: \"b221b02b-3291-4508-aa51-e60aa7533729\") "
Sep 05 08:05:45.262335 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.262292    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b221b02b-3291-4508-aa51-e60aa7533729-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b221b02b-3291-4508-aa51-e60aa7533729" (UID: "b221b02b-3291-4508-aa51-e60aa7533729"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:05:45.262497 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.262479    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b221b02b-3291-4508-aa51-e60aa7533729-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:45.264371 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.264331    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b221b02b-3291-4508-aa51-e60aa7533729-kube-api-access-d89mq" (OuterVolumeSpecName: "kube-api-access-d89mq") pod "b221b02b-3291-4508-aa51-e60aa7533729" (UID: "b221b02b-3291-4508-aa51-e60aa7533729"). InnerVolumeSpecName "kube-api-access-d89mq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:05:45.264465 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.264431    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b221b02b-3291-4508-aa51-e60aa7533729" (UID: "b221b02b-3291-4508-aa51-e60aa7533729"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:05:45.264680 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.264651    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b221b02b-3291-4508-aa51-e60aa7533729" (UID: "b221b02b-3291-4508-aa51-e60aa7533729"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:05:45.363633 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.363603    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:45.363633 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.363629    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d89mq\" (UniqueName: \"kubernetes.io/projected/b221b02b-3291-4508-aa51-e60aa7533729-kube-api-access-d89mq\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:45.363633 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.363638    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b221b02b-3291-4508-aa51-e60aa7533729-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:45.518854 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.518826    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"]
Sep 05 08:05:45.530623 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.530596    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-66998bbc-7b6d7658ff-bssf2"]
Sep 05 08:05:45.571671 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:45.571644    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b221b02b-3291-4508-aa51-e60aa7533729" path="/var/lib/kubelet/pods/b221b02b-3291-4508-aa51-e60aa7533729/volumes"
Sep 05 08:05:52.501835 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:52.501803    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"]
Sep 05 08:05:52.502321 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:52.502056    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy" containerID="cri-o://b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8" gracePeriod=30
Sep 05 08:05:56.773822 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:56.773784    2569 patch_prober.go:28] interesting pod/e2e-mt-66998bbc-openshift-default-6569645764-gczhs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 05 08:05:56.774176 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:56.773853    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 05 08:05:57.765981 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.765924    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:05:57.888344 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888314    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fk9b\" (UniqueName: \"kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-kube-api-access-6fk9b\") pod \"eb7cc9d9-598f-4370-ad98-fc7710f74791\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") "
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888373    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-data\") pod \"eb7cc9d9-598f-4370-ad98-fc7710f74791\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") "
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888409    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-envoy\") pod \"eb7cc9d9-598f-4370-ad98-fc7710f74791\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") "
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888456    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-podinfo\") pod \"eb7cc9d9-598f-4370-ad98-fc7710f74791\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") "
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888490    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-credential-socket\") pod \"eb7cc9d9-598f-4370-ad98-fc7710f74791\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") "
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888580    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb7cc9d9-598f-4370-ad98-fc7710f74791-istiod-ca-cert\") pod \"eb7cc9d9-598f-4370-ad98-fc7710f74791\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") "
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888618    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-socket\") pod \"eb7cc9d9-598f-4370-ad98-fc7710f74791\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") "
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888672    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-token\") pod \"eb7cc9d9-598f-4370-ad98-fc7710f74791\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") "
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888700    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-certs\") pod \"eb7cc9d9-598f-4370-ad98-fc7710f74791\" (UID: \"eb7cc9d9-598f-4370-ad98-fc7710f74791\") "
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888767    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb7cc9d9-598f-4370-ad98-fc7710f74791" (UID: "eb7cc9d9-598f-4370-ad98-fc7710f74791"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888822    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb7cc9d9-598f-4370-ad98-fc7710f74791" (UID: "eb7cc9d9-598f-4370-ad98-fc7710f74791"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:57.889033 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.888938    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7cc9d9-598f-4370-ad98-fc7710f74791-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb7cc9d9-598f-4370-ad98-fc7710f74791" (UID: "eb7cc9d9-598f-4370-ad98-fc7710f74791"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:05:57.889608 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.889106    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:57.889608 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.889124    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb7cc9d9-598f-4370-ad98-fc7710f74791-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:57.889608 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.889138    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:57.889608 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.889142    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb7cc9d9-598f-4370-ad98-fc7710f74791" (UID: "eb7cc9d9-598f-4370-ad98-fc7710f74791"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:57.889608 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.889399    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb7cc9d9-598f-4370-ad98-fc7710f74791" (UID: "eb7cc9d9-598f-4370-ad98-fc7710f74791"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:57.890836 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.890809    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb7cc9d9-598f-4370-ad98-fc7710f74791" (UID: "eb7cc9d9-598f-4370-ad98-fc7710f74791"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:05:57.891084 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.891065    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb7cc9d9-598f-4370-ad98-fc7710f74791" (UID: "eb7cc9d9-598f-4370-ad98-fc7710f74791"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:05:57.891287 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.891268    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-kube-api-access-6fk9b" (OuterVolumeSpecName: "kube-api-access-6fk9b") pod "eb7cc9d9-598f-4370-ad98-fc7710f74791" (UID: "eb7cc9d9-598f-4370-ad98-fc7710f74791"). InnerVolumeSpecName "kube-api-access-6fk9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:05:57.891287 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.891272    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb7cc9d9-598f-4370-ad98-fc7710f74791" (UID: "eb7cc9d9-598f-4370-ad98-fc7710f74791"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:05:57.990286 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.990233    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:57.990286 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.990280    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:57.990286 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.990289    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:57.990286 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.990298    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:57.990286 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.990306    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fk9b\" (UniqueName: \"kubernetes.io/projected/eb7cc9d9-598f-4370-ad98-fc7710f74791-kube-api-access-6fk9b\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:57.990630 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:57.990315    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb7cc9d9-598f-4370-ad98-fc7710f74791-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:05:58.051132 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.051049    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"]
Sep 05 08:05:58.051318 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.051291    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy" containerID="cri-o://1a4c563f63aa8a0ecd5c5b94e175133bcc7ada73ea5967d94f2717f16f52cb81" gracePeriod=30
Sep 05 08:05:58.267585 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.267544    2569 generic.go:356] "Generic (PLEG): container finished" podID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerID="b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8" exitCode=0
Sep 05 08:05:58.267585 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.267588    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" event={"ID":"eb7cc9d9-598f-4370-ad98-fc7710f74791","Type":"ContainerDied","Data":"b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8"}
Sep 05 08:05:58.267840 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.267600    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"
Sep 05 08:05:58.267840 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.267630    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs" event={"ID":"eb7cc9d9-598f-4370-ad98-fc7710f74791","Type":"ContainerDied","Data":"50dcb0589d2304ec68a428fca027c454e9599add4c3e19c66caf46e61b213ea5"}
Sep 05 08:05:58.267840 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.267651    2569 scope.go:117] "RemoveContainer" containerID="b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8"
Sep 05 08:05:58.277820 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.277802    2569 scope.go:117] "RemoveContainer" containerID="b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a"
Sep 05 08:05:58.286187 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.286171    2569 scope.go:117] "RemoveContainer" containerID="b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8"
Sep 05 08:05:58.286443 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:05:58.286424    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8\": container with ID starting with b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8 not found: ID does not exist" containerID="b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8"
Sep 05 08:05:58.286484 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.286451    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8"} err="failed to get container status \"b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8\": rpc error: code = NotFound desc = could not find container \"b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8\": container with ID starting with b57b99423dbaa9641b5e13730afb7cb5bac8b5ff22b0ebad08078090af3faaf8 not found: ID does not exist"
Sep 05 08:05:58.286484 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.286468    2569 scope.go:117] "RemoveContainer" containerID="b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a"
Sep 05 08:05:58.286702 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:05:58.286684    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a\": container with ID starting with b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a not found: ID does not exist" containerID="b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a"
Sep 05 08:05:58.286744 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.286712    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a"} err="failed to get container status \"b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a\": rpc error: code = NotFound desc = could not find container \"b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a\": container with ID starting with b4519f439625c89a2d76e3b27801bb2c060793875cf946a783ada92fb898e27a not found: ID does not exist"
Sep 05 08:05:58.306475 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.306415    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"]
Sep 05 08:05:58.322666 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:58.322643    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-66998bbc-openshift-default-6569645764-gczhs"]
Sep 05 08:05:59.574282 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:05:59.574249    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" path="/var/lib/kubelet/pods/eb7cc9d9-598f-4370-ad98-fc7710f74791/volumes"
Sep 05 08:06:03.299979 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.299945    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerID="1a4c563f63aa8a0ecd5c5b94e175133bcc7ada73ea5967d94f2717f16f52cb81" exitCode=0
Sep 05 08:06:03.300429 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.299998    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" event={"ID":"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304","Type":"ContainerDied","Data":"1a4c563f63aa8a0ecd5c5b94e175133bcc7ada73ea5967d94f2717f16f52cb81"}
Sep 05 08:06:03.300429 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.300054    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg" event={"ID":"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304","Type":"ContainerDied","Data":"203f120a5d503fe4afe3a0f2388cf25dec14b2b7e3a0f96dc0ddd89cb2defe93"}
Sep 05 08:06:03.300429 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.300074    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="203f120a5d503fe4afe3a0f2388cf25dec14b2b7e3a0f96dc0ddd89cb2defe93"
Sep 05 08:06:03.300429 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.300089    2569 scope.go:117] "RemoveContainer" containerID="a1a01ad20ba71d7da197d66b1399f7249e2fbbe1a4e5b0a13e4284b6e581f1eb"
Sep 05 08:06:03.303937 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.303916    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:06:03.449105 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449074    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-certs\") pod \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") "
Sep 05 08:06:03.449312 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449135    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-credential-socket\") pod \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") "
Sep 05 08:06:03.449312 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449154    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-socket\") pod \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") "
Sep 05 08:06:03.449312 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449186    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-token\") pod \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") "
Sep 05 08:06:03.449312 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449222    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-data\") pod \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") "
Sep 05 08:06:03.449312 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449248    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istiod-ca-cert\") pod \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") "
Sep 05 08:06:03.449312 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449273    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94qld\" (UniqueName: \"kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-kube-api-access-94qld\") pod \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") "
Sep 05 08:06:03.449661 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449363    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-podinfo\") pod \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") "
Sep 05 08:06:03.449661 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449377    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" (UID: "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:03.449661 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449396    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-envoy\") pod \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\" (UID: \"4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304\") "
Sep 05 08:06:03.449806 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449646    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" (UID: "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:03.449806 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449760    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:03.449806 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449781    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:03.449946 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449872    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" (UID: "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:03.449946 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.449932    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" (UID: "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:03.450169 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.450147    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" (UID: "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:06:03.452057 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.452024    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" (UID: "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:03.452057 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.452038    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-kube-api-access-94qld" (OuterVolumeSpecName: "kube-api-access-94qld") pod "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" (UID: "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304"). InnerVolumeSpecName "kube-api-access-94qld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:06:03.452057 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.452041    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" (UID: "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:06:03.452225 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.452107    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" (UID: "4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:06:03.550710 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.550674    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:03.550710 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.550701    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:03.550710 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.550711    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:03.550710 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.550719    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:03.550979 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.550727    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:03.550979 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.550735    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94qld\" (UniqueName: \"kubernetes.io/projected/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-kube-api-access-94qld\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:03.550979 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.550743    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:03.730635 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.730561    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"]
Sep 05 08:06:03.730818 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:03.730779    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf" podUID="0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" containerName="istio-proxy" containerID="cri-o://eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c" gracePeriod=30
Sep 05 08:06:04.306088 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:04.306060    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"
Sep 05 08:06:04.351111 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:04.351076    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"]
Sep 05 08:06:04.355625 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:04.355601    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e7c0ff-openshift-default-6cd6c4cc68-sf2hg"]
Sep 05 08:06:05.571956 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:05.571914    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" path="/var/lib/kubelet/pods/4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304/volumes"
Sep 05 08:06:07.258955 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.258918    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"]
Sep 05 08:06:07.259653 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259632    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy"
Sep 05 08:06:07.259725 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259657    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy"
Sep 05 08:06:07.259725 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259672    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy"
Sep 05 08:06:07.259725 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259680    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy"
Sep 05 08:06:07.259725 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259691    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b221b02b-3291-4508-aa51-e60aa7533729" containerName="maas-api"
Sep 05 08:06:07.259725 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259699    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b221b02b-3291-4508-aa51-e60aa7533729" containerName="maas-api"
Sep 05 08:06:07.259725 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259717    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy"
Sep 05 08:06:07.259725 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259725    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy"
Sep 05 08:06:07.259958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259735    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy"
Sep 05 08:06:07.259958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259744    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy"
Sep 05 08:06:07.259958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259760    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy"
Sep 05 08:06:07.259958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259767    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy"
Sep 05 08:06:07.259958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259908    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy"
Sep 05 08:06:07.259958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259918    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b221b02b-3291-4508-aa51-e60aa7533729" containerName="maas-api"
Sep 05 08:06:07.259958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259930    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy"
Sep 05 08:06:07.259958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259938    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9e4a1d-13c5-4bef-ab2d-ab6775fa0304" containerName="istio-proxy"
Sep 05 08:06:07.259958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259950    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy"
Sep 05 08:06:07.259958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.259959    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy"
Sep 05 08:06:07.263878 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.263861    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.266792 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.266772    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-8b6928-openshift-default-dockercfg-9mqlz\""
Sep 05 08:06:07.274313 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.274288    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"]
Sep 05 08:06:07.390865 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.390830    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4hpz\" (UniqueName: \"kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-kube-api-access-m4hpz\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.391054 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.390879    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-certs\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.391054 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.390966    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a0dc85-80c0-4342-b877-e48469c83f12-istiod-ca-cert\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.391054 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.390998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-envoy\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.391230 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.391084    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-socket\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.391230 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.391112    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-istio-token\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.391230 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.391205    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-credential-socket\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.391365 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.391230    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a0dc85-80c0-4342-b877-e48469c83f12-istio-podinfo\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.391365 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.391259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-data\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492108 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492075    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4hpz\" (UniqueName: \"kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-kube-api-access-m4hpz\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492108 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492114    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-certs\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492315 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492239    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a0dc85-80c0-4342-b877-e48469c83f12-istiod-ca-cert\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492315 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492281    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-envoy\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492389 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492360    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-socket\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492389 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492389    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-istio-token\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492519 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492468    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-credential-socket\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492519 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492492    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a0dc85-80c0-4342-b877-e48469c83f12-istio-podinfo\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492519 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492498    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-certs\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492725 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492556    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-data\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492798 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492722    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-socket\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492922 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492902    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-credential-socket\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.492985 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.492924    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-data\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.493251 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.493192    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a0dc85-80c0-4342-b877-e48469c83f12-istiod-ca-cert\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.494689 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.494670    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-envoy\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.494996 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.494979    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a0dc85-80c0-4342-b877-e48469c83f12-istio-podinfo\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.501401 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.501373    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4hpz\" (UniqueName: \"kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-kube-api-access-m4hpz\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.502071 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.502051    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-istio-token\") pod \"e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") " pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.575866 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.575843    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:07.854959 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.854922    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"]
Sep 05 08:06:07.856735 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:06:07.856709    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a0dc85_80c0_4342_b877_e48469c83f12.slice/crio-6d8c8428170b4e51f00c64f945e5c695c2eee32f1786d698d877914876b62185 WatchSource:0}: Error finding container 6d8c8428170b4e51f00c64f945e5c695c2eee32f1786d698d877914876b62185: Status 404 returned error can't find the container with id 6d8c8428170b4e51f00c64f945e5c695c2eee32f1786d698d877914876b62185
Sep 05 08:06:07.859252 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.859219    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:07.859352 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.859285    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:07.859352 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:07.859311    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:08.330680 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:08.330646    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" event={"ID":"73a0dc85-80c0-4342-b877-e48469c83f12","Type":"ContainerStarted","Data":"ccd13be5d8caccbabd4febb4599a851a4af9e2d55d52a46df68159f2035f9ec4"}
Sep 05 08:06:08.331051 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:08.330707    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" event={"ID":"73a0dc85-80c0-4342-b877-e48469c83f12","Type":"ContainerStarted","Data":"6d8c8428170b4e51f00c64f945e5c695c2eee32f1786d698d877914876b62185"}
Sep 05 08:06:08.356625 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:08.356574    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" podStartSLOduration=1.35655741 podStartE2EDuration="1.35655741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:06:08.353639558 +0000 UTC m=+1049.298961364" watchObservedRunningTime="2026-09-05 08:06:08.35655741 +0000 UTC m=+1049.301879215"
Sep 05 08:06:08.576281 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:08.576237    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:08.581284 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:08.581206    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:08.980568 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:08.980546    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:06:09.006368 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.006344    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-envoy\") pod \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") "
Sep 05 08:06:09.006542 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.006378    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-credential-socket\") pod \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") "
Sep 05 08:06:09.006542 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.006454    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-podinfo\") pod \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") "
Sep 05 08:06:09.006542 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.006481    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-data\") pod \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") "
Sep 05 08:06:09.006542 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.006515    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-socket\") pod \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") "
Sep 05 08:06:09.006753 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.006557    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-certs\") pod \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") "
Sep 05 08:06:09.006753 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.006583    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istiod-ca-cert\") pod \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") "
Sep 05 08:06:09.006753 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.006608    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7xpt\" (UniqueName: \"kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-kube-api-access-x7xpt\") pod \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") "
Sep 05 08:06:09.006753 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.006629    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-token\") pod \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\" (UID: \"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c\") "
Sep 05 08:06:09.007006 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.006937    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" (UID: "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:09.007173 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.007151    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-data" (OuterVolumeSpecName: "istio-data") pod "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" (UID: "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:09.007587 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.007338    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" (UID: "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:09.007689 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.007610    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" (UID: "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:09.007689 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.007673    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" (UID: "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:06:09.009252 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.009216    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" (UID: "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:06:09.010461 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.010436    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" (UID: "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:06:09.010461 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.010449    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-kube-api-access-x7xpt" (OuterVolumeSpecName: "kube-api-access-x7xpt") pod "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" (UID: "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c"). InnerVolumeSpecName "kube-api-access-x7xpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:06:09.010622 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.010470    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-token" (OuterVolumeSpecName: "istio-token") pod "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" (UID: "0d50c1b9-572b-4bc0-8bbc-fbe7e460865c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:06:09.108235 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.108149    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:09.108235 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.108180    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:09.108235 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.108195    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:09.108235 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.108207    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:09.108235 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.108220    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:09.108235 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.108232    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:09.108671 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.108255    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:09.108671 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.108268    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:09.108671 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.108281    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7xpt\" (UniqueName: \"kubernetes.io/projected/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c-kube-api-access-x7xpt\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:06:09.337187 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.337144    2569 generic.go:356] "Generic (PLEG): container finished" podID="0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" containerID="eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c" exitCode=0
Sep 05 08:06:09.337670 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.337195    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"
Sep 05 08:06:09.337670 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.337225    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf" event={"ID":"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c","Type":"ContainerDied","Data":"eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c"}
Sep 05 08:06:09.337670 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.337291    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf" event={"ID":"0d50c1b9-572b-4bc0-8bbc-fbe7e460865c","Type":"ContainerDied","Data":"62cb9939b39ace3632e6d7bf569300d6bf28074020b3b4388417fe78e9693169"}
Sep 05 08:06:09.337670 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.337325    2569 scope.go:117] "RemoveContainer" containerID="eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c"
Sep 05 08:06:09.337899 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.337883    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:09.339118 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.339097    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:09.347860 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.347824    2569 scope.go:117] "RemoveContainer" containerID="eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c"
Sep 05 08:06:09.348092 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:06:09.348075    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c\": container with ID starting with eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c not found: ID does not exist" containerID="eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c"
Sep 05 08:06:09.348139 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.348101    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c"} err="failed to get container status \"eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c\": rpc error: code = NotFound desc = could not find container \"eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c\": container with ID starting with eb893ec3dde4109437344dbf06276c60f01f911dd7c7044f538822f2c0308d5c not found: ID does not exist"
Sep 05 08:06:09.391612 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.387579    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"]
Sep 05 08:06:09.400333 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.400299    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-ecd628-openshift-default-5bc8c5cb84-47rrf"]
Sep 05 08:06:09.571739 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:09.571698    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" path="/var/lib/kubelet/pods/0d50c1b9-572b-4bc0-8bbc-fbe7e460865c/volumes"
Sep 05 08:06:17.089264 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.089232    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"]
Sep 05 08:06:17.089902 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.089879    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" containerName="istio-proxy"
Sep 05 08:06:17.089991 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.089904    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" containerName="istio-proxy"
Sep 05 08:06:17.089991 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.089960    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy"
Sep 05 08:06:17.089991 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.089969    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb7cc9d9-598f-4370-ad98-fc7710f74791" containerName="istio-proxy"
Sep 05 08:06:17.090142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.090072    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="64221411-fade-4538-a5db-b9916919dc2c" containerName="istio-proxy"
Sep 05 08:06:17.090142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.090086    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d50c1b9-572b-4bc0-8bbc-fbe7e460865c" containerName="istio-proxy"
Sep 05 08:06:17.094723 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.094702    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.097740 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.097718    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-8c2e20-openshift-default-dockercfg-r2wt8\""
Sep 05 08:06:17.105546 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.105491    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"]
Sep 05 08:06:17.185387 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.185351    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-socket\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.185605 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.185398    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-podinfo\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.185605 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.185416    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv5xs\" (UniqueName: \"kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-kube-api-access-pv5xs\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.185605 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.185575    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-envoy\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.185816 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.185648    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d06e4fd8-2449-4803-a7ab-fc71b8430625-istiod-ca-cert\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.185816 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.185714    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-data\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.185816 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.185763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-certs\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.185816 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.185801    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-token\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.185993 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.185828    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-credential-socket\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.286987 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.286947    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-envoy\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.287190 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.287001    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d06e4fd8-2449-4803-a7ab-fc71b8430625-istiod-ca-cert\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.287190 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.287025    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-data\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.287190 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.287050    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-certs\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.287378 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.287203    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-token\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.287378 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.287309    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-credential-socket\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.287485 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.287429    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-socket\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.287485 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.287476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-podinfo\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.287627 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.287483    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-data\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.287627 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.287504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv5xs\" (UniqueName: \"kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-kube-api-access-pv5xs\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.287979 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.287926    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-certs\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.288109 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.288054    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-credential-socket\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.288181 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.288113    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-socket\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.288256 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.288215    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d06e4fd8-2449-4803-a7ab-fc71b8430625-istiod-ca-cert\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.289609 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.289590    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-envoy\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.290256 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.290239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-podinfo\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.296302 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.296282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-token\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.296398 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.296322    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv5xs\" (UniqueName: \"kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-kube-api-access-pv5xs\") pod \"e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") " pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.406978 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.406888    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:17.718772 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.718744    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"]
Sep 05 08:06:17.721142 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:06:17.721112    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06e4fd8_2449_4803_a7ab_fc71b8430625.slice/crio-d56b1f3a93a71abcd65699a3779b9f1b19113b864b77af49b9aae5f84db0030f WatchSource:0}: Error finding container d56b1f3a93a71abcd65699a3779b9f1b19113b864b77af49b9aae5f84db0030f: Status 404 returned error can't find the container with id d56b1f3a93a71abcd65699a3779b9f1b19113b864b77af49b9aae5f84db0030f
Sep 05 08:06:17.723757 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.723715    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:17.723850 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.723812    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:17.723902 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:17.723858    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:18.396113 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:18.396078    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" event={"ID":"d06e4fd8-2449-4803-a7ab-fc71b8430625","Type":"ContainerStarted","Data":"8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db"}
Sep 05 08:06:18.396634 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:18.396132    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" event={"ID":"d06e4fd8-2449-4803-a7ab-fc71b8430625","Type":"ContainerStarted","Data":"d56b1f3a93a71abcd65699a3779b9f1b19113b864b77af49b9aae5f84db0030f"}
Sep 05 08:06:18.407064 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:18.407040    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:18.412592 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:18.412572    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:18.475090 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:18.475033    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" podStartSLOduration=1.475017576 podStartE2EDuration="1.475017576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:06:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:06:18.447725922 +0000 UTC m=+1059.393047739" watchObservedRunningTime="2026-09-05 08:06:18.475017576 +0000 UTC m=+1059.420339381"
Sep 05 08:06:19.401983 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:19.401949    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:19.402783 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:19.402764    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:25.379701 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:25.379657    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 08:06:25.380167 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:25.379727    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 08:06:26.444127 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:26.444089    2569 generic.go:356] "Generic (PLEG): container finished" podID="73a0dc85-80c0-4342-b877-e48469c83f12" containerID="ccd13be5d8caccbabd4febb4599a851a4af9e2d55d52a46df68159f2035f9ec4" exitCode=0
Sep 05 08:06:26.444579 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:26.444161    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" event={"ID":"73a0dc85-80c0-4342-b877-e48469c83f12","Type":"ContainerDied","Data":"ccd13be5d8caccbabd4febb4599a851a4af9e2d55d52a46df68159f2035f9ec4"}
Sep 05 08:06:26.444637 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:26.444586    2569 scope.go:117] "RemoveContainer" containerID="ccd13be5d8caccbabd4febb4599a851a4af9e2d55d52a46df68159f2035f9ec4"
Sep 05 08:06:26.445744 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:26.445714    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:26.445884 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:26.445779    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:26.445884 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:26.445807    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:27.455588 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:27.455550    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" event={"ID":"73a0dc85-80c0-4342-b877-e48469c83f12","Type":"ContainerStarted","Data":"96d0196611fced916637214ad47f205cadf36934f17cec7b168bc5830bb165fa"}
Sep 05 08:06:27.577400 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:27.577360    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:27.577628 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:27.577487    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 05 08:06:27.577628 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:27.577554    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 05 08:06:28.576792 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:28.576738    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 05 08:06:28.577361 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:28.576830    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 05 08:06:29.577225 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:29.577187    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 05 08:06:29.577843 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:29.577809    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 05 08:06:30.576819 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:30.576724    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 05 08:06:30.577087 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:30.577058    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 05 08:06:31.577403 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:31.577366    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 05 08:06:31.577908 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:31.577432    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 05 08:06:32.576506 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:32.576474    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 05 08:06:32.576677 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:32.576550    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 05 08:06:33.580880 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:33.580844    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:33.581382 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:33.581365    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:33.582141 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:33.582117    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:06:35.442696 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:35.442653    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 08:06:35.443127 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:35.442711    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 08:06:35.998708 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:35.998680    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"]
Sep 05 08:06:36.001153 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.001136    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.005037 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.005012    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-cec446-openshift-default-dockercfg-gltwp\""
Sep 05 08:06:36.045345 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.045195    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"]
Sep 05 08:06:36.097789 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.097751    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-socket\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.097986 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.097795    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-credential-socket\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.097986 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.097849    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-certs\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.097986 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.097870    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-data\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.097986 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.097951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a8c9e09-cd2b-497d-b0bc-33067258136c-istiod-ca-cert\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.098215 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.098008    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cgkd\" (UniqueName: \"kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-kube-api-access-5cgkd\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.098215 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.098045    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-token\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.098215 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.098086    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-envoy\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.098215 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.098135    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-podinfo\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199183 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199148    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a8c9e09-cd2b-497d-b0bc-33067258136c-istiod-ca-cert\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199347 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cgkd\" (UniqueName: \"kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-kube-api-access-5cgkd\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199347 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199221    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-token\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199347 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-envoy\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199347 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-podinfo\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199347 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199309    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-socket\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199347 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199330    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-credential-socket\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199712 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-certs\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199712 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199385    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-data\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199827 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199738    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-socket\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199827 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199784    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-data\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.199827 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-credential-socket\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.200017 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.199999    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-certs\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.200017 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.200009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a8c9e09-cd2b-497d-b0bc-33067258136c-istiod-ca-cert\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.201701 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.201684    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-envoy\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.202024 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.202005    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-podinfo\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.211729 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.211706    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-token\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.212113 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.212099    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cgkd\" (UniqueName: \"kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-kube-api-access-5cgkd\") pod \"e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") " pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.311927 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.311860    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:36.521779 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.521751    2569 generic.go:356] "Generic (PLEG): container finished" podID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerID="8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db" exitCode=0
Sep 05 08:06:36.522163 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.521826    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" event={"ID":"d06e4fd8-2449-4803-a7ab-fc71b8430625","Type":"ContainerDied","Data":"8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db"}
Sep 05 08:06:36.522259 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.522245    2569 scope.go:117] "RemoveContainer" containerID="8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db"
Sep 05 08:06:36.523342 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.523316    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:36.523418 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.523399    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:36.523457 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.523444    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:36.597554 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.597512    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"]
Sep 05 08:06:36.599296 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:06:36.599265    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8c9e09_cd2b_497d_b0bc_33067258136c.slice/crio-812f1649439ea0682aab65c40d003d054ebd83e912df3038abb23e9e80822636 WatchSource:0}: Error finding container 812f1649439ea0682aab65c40d003d054ebd83e912df3038abb23e9e80822636: Status 404 returned error can't find the container with id 812f1649439ea0682aab65c40d003d054ebd83e912df3038abb23e9e80822636
Sep 05 08:06:36.601961 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.601915    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:36.602069 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.601989    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:36.602069 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:36.602030    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:37.528971 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:37.528940    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" event={"ID":"d06e4fd8-2449-4803-a7ab-fc71b8430625","Type":"ContainerStarted","Data":"afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78"}
Sep 05 08:06:37.530609 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:37.530583    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" event={"ID":"0a8c9e09-cd2b-497d-b0bc-33067258136c","Type":"ContainerStarted","Data":"7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c"}
Sep 05 08:06:37.530730 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:37.530626    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" event={"ID":"0a8c9e09-cd2b-497d-b0bc-33067258136c","Type":"ContainerStarted","Data":"812f1649439ea0682aab65c40d003d054ebd83e912df3038abb23e9e80822636"}
Sep 05 08:06:37.575383 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:37.575330    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" podStartSLOduration=2.575313655 podStartE2EDuration="2.575313655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:06:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:06:37.573849585 +0000 UTC m=+1078.519171392" watchObservedRunningTime="2026-09-05 08:06:37.575313655 +0000 UTC m=+1078.520635530"
Sep 05 08:06:38.312336 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:38.312301    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:38.317233 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:38.317206    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:38.407165 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:38.407131    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:38.407338 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:38.407280    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 05 08:06:38.407424 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:38.407339    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 05 08:06:38.536372 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:38.536344    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:38.537250 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:38.537228    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:39.407503 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:39.407471    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 05 08:06:39.407707 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:39.407553    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 05 08:06:40.407868 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.407828    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 05 08:06:40.408331 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.407890    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 05 08:06:40.826330 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.826300    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"]
Sep 05 08:06:40.828977 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.828958    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:40.831913 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.831877    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-cec446-serving-cert\""
Sep 05 08:06:40.832069 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.832048    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c62fz\""
Sep 05 08:06:40.842760 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.842727    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"]
Sep 05 08:06:40.959552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.955077    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9rxx\" (UniqueName: \"kubernetes.io/projected/da864b70-d5a4-4394-91ca-8de0964852b1-kube-api-access-g9rxx\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:40.959552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.955195    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:40.959552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.955234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:40.959552 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:40.955317    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da864b70-d5a4-4394-91ca-8de0964852b1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:41.056338 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.056295    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da864b70-d5a4-4394-91ca-8de0964852b1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:41.056549 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.056369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9rxx\" (UniqueName: \"kubernetes.io/projected/da864b70-d5a4-4394-91ca-8de0964852b1-kube-api-access-g9rxx\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:41.056549 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.056467    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:41.056549 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.056503    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:41.057115 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.057084    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da864b70-d5a4-4394-91ca-8de0964852b1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:41.057209 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:06:41.057169    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-serving-cert: secret "maas-api-e2e-shared-b-w2-cec446-serving-cert" not found
Sep 05 08:06:41.057274 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:06:41.057233    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-maas-api-tls podName:da864b70-d5a4-4394-91ca-8de0964852b1 nodeName:}" failed. No retries permitted until 2026-09-05 08:06:41.557214515 +0000 UTC m=+1082.502536313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-maas-api-tls") pod "maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5" (UID: "da864b70-d5a4-4394-91ca-8de0964852b1") : secret "maas-api-e2e-shared-b-w2-cec446-serving-cert" not found
Sep 05 08:06:41.059777 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.059749    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:41.066612 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.066586    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9rxx\" (UniqueName: \"kubernetes.io/projected/da864b70-d5a4-4394-91ca-8de0964852b1-kube-api-access-g9rxx\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:41.407367 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.407335    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 05 08:06:41.407617 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.407398    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 05 08:06:41.563629 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.563593    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:41.568044 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.568016    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:41.740309 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:41.740199    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:42.029727 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:42.029699    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"]
Sep 05 08:06:42.031095 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:06:42.031069    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda864b70_d5a4_4394_91ca_8de0964852b1.slice/crio-21f9a215733d2a7cf5a5cfa35317e2dce948c3f3549bac53fd255f0989a99998 WatchSource:0}: Error finding container 21f9a215733d2a7cf5a5cfa35317e2dce948c3f3549bac53fd255f0989a99998: Status 404 returned error can't find the container with id 21f9a215733d2a7cf5a5cfa35317e2dce948c3f3549bac53fd255f0989a99998
Sep 05 08:06:42.032778 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:42.032761    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 08:06:42.407452 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:42.407422    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 05 08:06:42.407604 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:42.407475    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 05 08:06:42.576164 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:42.576130    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5" event={"ID":"da864b70-d5a4-4394-91ca-8de0964852b1","Type":"ContainerStarted","Data":"940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580"}
Sep 05 08:06:42.576665 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:42.576184    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5" event={"ID":"da864b70-d5a4-4394-91ca-8de0964852b1","Type":"ContainerStarted","Data":"21f9a215733d2a7cf5a5cfa35317e2dce948c3f3549bac53fd255f0989a99998"}
Sep 05 08:06:42.576665 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:42.576264    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:42.596305 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:42.596243    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5" podStartSLOduration=2.26168059 podStartE2EDuration="2.596226335s" totalImagesPullingTime="334.545745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:06:42.595407468 +0000 UTC m=+1083.540729272" watchObservedRunningTime="2026-09-05 08:06:42.596226335 +0000 UTC m=+1083.541548139"
Sep 05 08:06:43.410829 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:43.410800    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:43.411183 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:43.411163    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:43.411845 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:43.411825    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:06:48.589057 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:48.589029    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:06:50.224264 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.224102    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"]
Sep 05 08:06:50.227325 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.227306    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.230257 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.230232    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-5ef325-serving-cert\""
Sep 05 08:06:50.246550 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.239249    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"]
Sep 05 08:06:50.272579 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.263423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65h6d\" (UniqueName: \"kubernetes.io/projected/7d87658e-9333-4948-aa74-637996ab1c4b-kube-api-access-65h6d\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.272579 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.263615    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.272579 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.263663    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.272579 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.263744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d87658e-9333-4948-aa74-637996ab1c4b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.364402 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.364319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65h6d\" (UniqueName: \"kubernetes.io/projected/7d87658e-9333-4948-aa74-637996ab1c4b-kube-api-access-65h6d\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.364637 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.364436    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.364637 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.364479    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.364637 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.364538    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d87658e-9333-4948-aa74-637996ab1c4b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.365397 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.365353    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d87658e-9333-4948-aa74-637996ab1c4b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.367631 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.367598    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.367751 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.367659    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.374941 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.374919    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65h6d\" (UniqueName: \"kubernetes.io/projected/7d87658e-9333-4948-aa74-637996ab1c4b-kube-api-access-65h6d\") pod \"maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.539479 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.539377    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:50.855555 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:50.854660    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"]
Sep 05 08:06:50.859103 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:06:50.859070    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d87658e_9333_4948_aa74_637996ab1c4b.slice/crio-33ca5cafab9ad410166a31ff48379e7f75ac2e0dbf0c34b74cf6f20aa3ea4237 WatchSource:0}: Error finding container 33ca5cafab9ad410166a31ff48379e7f75ac2e0dbf0c34b74cf6f20aa3ea4237: Status 404 returned error can't find the container with id 33ca5cafab9ad410166a31ff48379e7f75ac2e0dbf0c34b74cf6f20aa3ea4237
Sep 05 08:06:51.634510 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:51.634470    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv" event={"ID":"7d87658e-9333-4948-aa74-637996ab1c4b","Type":"ContainerStarted","Data":"dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24"}
Sep 05 08:06:51.634887 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:51.634542    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv" event={"ID":"7d87658e-9333-4948-aa74-637996ab1c4b","Type":"ContainerStarted","Data":"33ca5cafab9ad410166a31ff48379e7f75ac2e0dbf0c34b74cf6f20aa3ea4237"}
Sep 05 08:06:51.634887 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:51.634676    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:51.655049 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:51.654969    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv" podStartSLOduration=1.240375302 podStartE2EDuration="1.654949224s" totalImagesPullingTime="414.573922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:06:51.65179414 +0000 UTC m=+1092.597115937" watchObservedRunningTime="2026-09-05 08:06:51.654949224 +0000 UTC m=+1092.600271021"
Sep 05 08:06:54.577680 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:54.577638    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 08:06:54.578054 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:54.577689    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 08:06:56.664570 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:56.664510    2569 generic.go:356] "Generic (PLEG): container finished" podID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerID="7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c" exitCode=0
Sep 05 08:06:56.664969 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:56.664574    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" event={"ID":"0a8c9e09-cd2b-497d-b0bc-33067258136c","Type":"ContainerDied","Data":"7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c"}
Sep 05 08:06:56.664969 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:56.664945    2569 scope.go:117] "RemoveContainer" containerID="7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c"
Sep 05 08:06:56.666135 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:56.666109    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:56.666190 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:56.666180    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:56.666226 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:56.666210    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:06:57.655576 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:57.655543    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:06:57.672756 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:57.672723    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" event={"ID":"0a8c9e09-cd2b-497d-b0bc-33067258136c","Type":"ContainerStarted","Data":"9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390"}
Sep 05 08:06:58.312939 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:58.312904    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:06:58.313165 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:58.313135    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 05 08:06:58.313248 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:58.313197    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 05 08:06:59.312185 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:59.312155    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 05 08:06:59.312576 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:06:59.312205    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 05 08:07:00.312545 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:00.312491    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 05 08:07:00.312942 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:00.312591    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 05 08:07:01.312444 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:01.312405    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 05 08:07:01.312641 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:01.312457    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 05 08:07:02.312620 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:02.312585    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 05 08:07:02.312806 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:02.312650    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 05 08:07:03.317504 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:03.317473    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:07:03.317976 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:03.317888    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:07:03.318505 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:03.318487    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:07:22.232470 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.232437    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"]
Sep 05 08:07:22.237100 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.237078    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.240146 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.240124    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4b62745d-serving-cert\""
Sep 05 08:07:22.246929 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.246907    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"]
Sep 05 08:07:22.398069 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.398035    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnj22\" (UniqueName: \"kubernetes.io/projected/d6c81253-a66a-4a97-b84c-6a85194f58fb-kube-api-access-lnj22\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.398252 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.398091    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-metrics-certs\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.398252 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.398151    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-maas-api-tls\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.398252 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.398166    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c81253-a66a-4a97-b84c-6a85194f58fb-openshift-service-ca\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.499638 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.499565    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-maas-api-tls\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.499638 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.499595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c81253-a66a-4a97-b84c-6a85194f58fb-openshift-service-ca\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.499842 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.499665    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnj22\" (UniqueName: \"kubernetes.io/projected/d6c81253-a66a-4a97-b84c-6a85194f58fb-kube-api-access-lnj22\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.499842 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.499780    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-metrics-certs\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.500372 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.500345    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c81253-a66a-4a97-b84c-6a85194f58fb-openshift-service-ca\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.502101 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.502081    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-maas-api-tls\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.502191 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.502141    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-metrics-certs\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.511391 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.511368    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnj22\" (UniqueName: \"kubernetes.io/projected/d6c81253-a66a-4a97-b84c-6a85194f58fb-kube-api-access-lnj22\") pod \"maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.550383 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.550357    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:22.826724 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:22.826700    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"]
Sep 05 08:07:22.828035 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:07:22.828003    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c81253_a66a_4a97_b84c_6a85194f58fb.slice/crio-7cbe5955e1e35474f1e4c47a0382a9bfddc425ea564c6061024a8d87e0354821 WatchSource:0}: Error finding container 7cbe5955e1e35474f1e4c47a0382a9bfddc425ea564c6061024a8d87e0354821: Status 404 returned error can't find the container with id 7cbe5955e1e35474f1e4c47a0382a9bfddc425ea564c6061024a8d87e0354821
Sep 05 08:07:23.816575 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:23.816460    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz" event={"ID":"d6c81253-a66a-4a97-b84c-6a85194f58fb","Type":"ContainerStarted","Data":"93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af"}
Sep 05 08:07:23.816575 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:23.816516    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz" event={"ID":"d6c81253-a66a-4a97-b84c-6a85194f58fb","Type":"ContainerStarted","Data":"7cbe5955e1e35474f1e4c47a0382a9bfddc425ea564c6061024a8d87e0354821"}
Sep 05 08:07:23.816972 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:23.816599    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:23.837938 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:23.837891    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz" podStartSLOduration=1.321748759 podStartE2EDuration="1.837872074s" totalImagesPullingTime="516.123315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:07:23.835938519 +0000 UTC m=+1124.781260323" watchObservedRunningTime="2026-09-05 08:07:23.837872074 +0000 UTC m=+1124.783193880"
Sep 05 08:07:29.828397 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:29.828367    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:07:41.455614 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:07:41.455586    2569 scope.go:117] "RemoveContainer" containerID="1a4c563f63aa8a0ecd5c5b94e175133bcc7ada73ea5967d94f2717f16f52cb81"
Sep 05 08:08:39.606482 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:39.606454    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:08:39.611226 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:39.611206    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:08:46.845232 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:46.845186    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"]
Sep 05 08:08:46.845707 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:46.845431    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz" podUID="d6c81253-a66a-4a97-b84c-6a85194f58fb" containerName="maas-api" containerID="cri-o://93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af" gracePeriod=30
Sep 05 08:08:47.099938 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.099878    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:08:47.145818 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.145767    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-maas-api-tls\") pod \"d6c81253-a66a-4a97-b84c-6a85194f58fb\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") "
Sep 05 08:08:47.146043 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.145894    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnj22\" (UniqueName: \"kubernetes.io/projected/d6c81253-a66a-4a97-b84c-6a85194f58fb-kube-api-access-lnj22\") pod \"d6c81253-a66a-4a97-b84c-6a85194f58fb\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") "
Sep 05 08:08:47.146043 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.145933    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c81253-a66a-4a97-b84c-6a85194f58fb-openshift-service-ca\") pod \"d6c81253-a66a-4a97-b84c-6a85194f58fb\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") "
Sep 05 08:08:47.146043 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.146001    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-metrics-certs\") pod \"d6c81253-a66a-4a97-b84c-6a85194f58fb\" (UID: \"d6c81253-a66a-4a97-b84c-6a85194f58fb\") "
Sep 05 08:08:47.146384 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.146326    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6c81253-a66a-4a97-b84c-6a85194f58fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d6c81253-a66a-4a97-b84c-6a85194f58fb" (UID: "d6c81253-a66a-4a97-b84c-6a85194f58fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:08:47.148829 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.148789    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d6c81253-a66a-4a97-b84c-6a85194f58fb" (UID: "d6c81253-a66a-4a97-b84c-6a85194f58fb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:08:47.155440 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.155399    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6c81253-a66a-4a97-b84c-6a85194f58fb-kube-api-access-lnj22" (OuterVolumeSpecName: "kube-api-access-lnj22") pod "d6c81253-a66a-4a97-b84c-6a85194f58fb" (UID: "d6c81253-a66a-4a97-b84c-6a85194f58fb"). InnerVolumeSpecName "kube-api-access-lnj22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:08:47.155590 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.155498    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d6c81253-a66a-4a97-b84c-6a85194f58fb" (UID: "d6c81253-a66a-4a97-b84c-6a85194f58fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:08:47.247488 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.247449    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:08:47.247488 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.247482    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnj22\" (UniqueName: \"kubernetes.io/projected/d6c81253-a66a-4a97-b84c-6a85194f58fb-kube-api-access-lnj22\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:08:47.247488 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.247496    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c81253-a66a-4a97-b84c-6a85194f58fb-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:08:47.247833 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.247508    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6c81253-a66a-4a97-b84c-6a85194f58fb-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:08:47.275817 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.275782    2569 generic.go:356] "Generic (PLEG): container finished" podID="d6c81253-a66a-4a97-b84c-6a85194f58fb" containerID="93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af" exitCode=0
Sep 05 08:08:47.275998 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.275832    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"
Sep 05 08:08:47.275998 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.275839    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz" event={"ID":"d6c81253-a66a-4a97-b84c-6a85194f58fb","Type":"ContainerDied","Data":"93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af"}
Sep 05 08:08:47.275998 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.275892    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz" event={"ID":"d6c81253-a66a-4a97-b84c-6a85194f58fb","Type":"ContainerDied","Data":"7cbe5955e1e35474f1e4c47a0382a9bfddc425ea564c6061024a8d87e0354821"}
Sep 05 08:08:47.275998 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.275923    2569 scope.go:117] "RemoveContainer" containerID="93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af"
Sep 05 08:08:47.286319 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.286301    2569 scope.go:117] "RemoveContainer" containerID="93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af"
Sep 05 08:08:47.286603 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:08:47.286583    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af\": container with ID starting with 93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af not found: ID does not exist" containerID="93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af"
Sep 05 08:08:47.286705 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.286609    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af"} err="failed to get container status \"93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af\": rpc error: code = NotFound desc = could not find container \"93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af\": container with ID starting with 93bb13cba4ada6195840aeefbad17cd4b7c1fadbc2c9488181b1f69368c7d1af not found: ID does not exist"
Sep 05 08:08:47.301290 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.301267    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"]
Sep 05 08:08:47.306910 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.306890    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4b62745d-56b5596bf4-8zrxz"]
Sep 05 08:08:47.571445 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:08:47.571414    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6c81253-a66a-4a97-b84c-6a85194f58fb" path="/var/lib/kubelet/pods/d6c81253-a66a-4a97-b84c-6a85194f58fb/volumes"
Sep 05 08:10:51.397934 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.397889    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"]
Sep 05 08:10:51.398623 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.398469    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6c81253-a66a-4a97-b84c-6a85194f58fb" containerName="maas-api"
Sep 05 08:10:51.398623 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.398485    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6c81253-a66a-4a97-b84c-6a85194f58fb" containerName="maas-api"
Sep 05 08:10:51.398623 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.398607    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6c81253-a66a-4a97-b84c-6a85194f58fb" containerName="maas-api"
Sep 05 08:10:51.401889 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.401871    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.404908 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.404888    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-e171e4-serving-cert\""
Sep 05 08:10:51.414767 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.414738    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"]
Sep 05 08:10:51.516790 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.516747    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-openshift-service-ca\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.516982 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.516899    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-metrics-certs\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.516982 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.516952    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-maas-api-tls\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.516982 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.516972    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrksl\" (UniqueName: \"kubernetes.io/projected/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-kube-api-access-hrksl\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.618432 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.618399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-metrics-certs\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.618613 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.618451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-maas-api-tls\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.618613 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.618467    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrksl\" (UniqueName: \"kubernetes.io/projected/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-kube-api-access-hrksl\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.618613 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.618520    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-openshift-service-ca\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.618745 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:10:51.618616    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-serving-cert: secret "maas-api-e2e-api-a-e171e4-serving-cert" not found
Sep 05 08:10:51.618745 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:10:51.618695    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-maas-api-tls podName:fcc5e8b1-a638-435f-b790-9f13fdc2a45a nodeName:}" failed. No retries permitted until 2026-09-05 08:10:52.118672936 +0000 UTC m=+1333.063994730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-maas-api-tls") pod "maas-api-e2e-api-a-e171e4-8596655564-pr5s6" (UID: "fcc5e8b1-a638-435f-b790-9f13fdc2a45a") : secret "maas-api-e2e-api-a-e171e4-serving-cert" not found
Sep 05 08:10:51.619236 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.619213    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-openshift-service-ca\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.621213 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.621193    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-metrics-certs\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:51.627154 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:51.627133    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrksl\" (UniqueName: \"kubernetes.io/projected/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-kube-api-access-hrksl\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:52.124086 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:52.124049    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-maas-api-tls\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:52.126929 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:52.126902    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-maas-api-tls\") pod \"maas-api-e2e-api-a-e171e4-8596655564-pr5s6\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:52.315982 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:52.315930    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:52.591084 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:52.591057    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"]
Sep 05 08:10:52.592237 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:10:52.592215    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc5e8b1_a638_435f_b790_9f13fdc2a45a.slice/crio-879757135d2815c47fe0e6cd5e2afc9f963734b5889792d11f5919ef018197ef WatchSource:0}: Error finding container 879757135d2815c47fe0e6cd5e2afc9f963734b5889792d11f5919ef018197ef: Status 404 returned error can't find the container with id 879757135d2815c47fe0e6cd5e2afc9f963734b5889792d11f5919ef018197ef
Sep 05 08:10:52.982515 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:52.982426    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6" event={"ID":"fcc5e8b1-a638-435f-b790-9f13fdc2a45a","Type":"ContainerStarted","Data":"879757135d2815c47fe0e6cd5e2afc9f963734b5889792d11f5919ef018197ef"}
Sep 05 08:10:53.989905 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:53.989870    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6" event={"ID":"fcc5e8b1-a638-435f-b790-9f13fdc2a45a","Type":"ContainerStarted","Data":"2b494eace076645d0dccca9fd739f9ca5dee93c85bcf772941e81ac422c94847"}
Sep 05 08:10:53.990307 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:53.989933    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:10:54.010931 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:10:54.010878    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6" podStartSLOduration=2.584538644 podStartE2EDuration="3.010862552s" totalImagesPullingTime="426.323908ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:10:54.008590399 +0000 UTC m=+1334.953912204" watchObservedRunningTime="2026-09-05 08:10:54.010862552 +0000 UTC m=+1334.956184358"
Sep 05 08:11:00.002216 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:00.002184    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:11:20.600428 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.600388    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"]
Sep 05 08:11:20.604208 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.604187    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.607464 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.607123    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-d850e3-serving-cert\""
Sep 05 08:11:20.614729 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.614701    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"]
Sep 05 08:11:20.729259 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.729217    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtm4g\" (UniqueName: \"kubernetes.io/projected/e39a9c02-31a1-470f-a508-a002b62343e7-kube-api-access-gtm4g\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.729470 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.729353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39a9c02-31a1-470f-a508-a002b62343e7-openshift-service-ca\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.729565 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.729473    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-metrics-certs\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.729664 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.729628    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-maas-api-tls\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.830617 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.830580    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39a9c02-31a1-470f-a508-a002b62343e7-openshift-service-ca\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.830790 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.830656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-metrics-certs\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.830790 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.830710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-maas-api-tls\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.830790 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.830756    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtm4g\" (UniqueName: \"kubernetes.io/projected/e39a9c02-31a1-470f-a508-a002b62343e7-kube-api-access-gtm4g\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.830955 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:11:20.830885    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-serving-cert: secret "maas-api-e2e-api-b-d850e3-serving-cert" not found
Sep 05 08:11:20.831009 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:11:20.830977    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-maas-api-tls podName:e39a9c02-31a1-470f-a508-a002b62343e7 nodeName:}" failed. No retries permitted until 2026-09-05 08:11:21.33095231 +0000 UTC m=+1362.276274107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-maas-api-tls") pod "maas-api-e2e-api-b-d850e3-7d799944c-tfk4z" (UID: "e39a9c02-31a1-470f-a508-a002b62343e7") : secret "maas-api-e2e-api-b-d850e3-serving-cert" not found
Sep 05 08:11:20.831314 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.831280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39a9c02-31a1-470f-a508-a002b62343e7-openshift-service-ca\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.833289 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.833266    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-metrics-certs\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:20.842731 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:20.842703    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtm4g\" (UniqueName: \"kubernetes.io/projected/e39a9c02-31a1-470f-a508-a002b62343e7-kube-api-access-gtm4g\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:21.335378 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:21.335339    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-maas-api-tls\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:21.338062 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:21.338039    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-maas-api-tls\") pod \"maas-api-e2e-api-b-d850e3-7d799944c-tfk4z\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:21.519332 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:21.519281    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:21.802989 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:21.802962    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"]
Sep 05 08:11:21.804789 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:11:21.804759    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39a9c02_31a1_470f_a508_a002b62343e7.slice/crio-180da7087b0ea73a64326758fb8d1d0ad3054f04bb7ca90bd8663dba7f1a653e WatchSource:0}: Error finding container 180da7087b0ea73a64326758fb8d1d0ad3054f04bb7ca90bd8663dba7f1a653e: Status 404 returned error can't find the container with id 180da7087b0ea73a64326758fb8d1d0ad3054f04bb7ca90bd8663dba7f1a653e
Sep 05 08:11:22.161577 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:22.161518    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z" event={"ID":"e39a9c02-31a1-470f-a508-a002b62343e7","Type":"ContainerStarted","Data":"180da7087b0ea73a64326758fb8d1d0ad3054f04bb7ca90bd8663dba7f1a653e"}
Sep 05 08:11:23.168834 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:23.168792    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z" event={"ID":"e39a9c02-31a1-470f-a508-a002b62343e7","Type":"ContainerStarted","Data":"831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8"}
Sep 05 08:11:23.169355 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:23.168914    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:11:23.188926 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:23.188869    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z" podStartSLOduration=2.86308875 podStartE2EDuration="3.188853511s" totalImagesPullingTime="325.764761ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:11:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:11:23.187268311 +0000 UTC m=+1364.132590119" watchObservedRunningTime="2026-09-05 08:11:23.188853511 +0000 UTC m=+1364.134175316"
Sep 05 08:11:29.180513 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:11:29.180479    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:12:03.155833 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.155799    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"]
Sep 05 08:12:03.158638 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.156081    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6" podUID="fcc5e8b1-a638-435f-b790-9f13fdc2a45a" containerName="maas-api" containerID="cri-o://2b494eace076645d0dccca9fd739f9ca5dee93c85bcf772941e81ac422c94847" gracePeriod=30
Sep 05 08:12:03.408408 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.408383    2569 generic.go:356] "Generic (PLEG): container finished" podID="fcc5e8b1-a638-435f-b790-9f13fdc2a45a" containerID="2b494eace076645d0dccca9fd739f9ca5dee93c85bcf772941e81ac422c94847" exitCode=0
Sep 05 08:12:03.408497 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.408464    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6" event={"ID":"fcc5e8b1-a638-435f-b790-9f13fdc2a45a","Type":"ContainerDied","Data":"2b494eace076645d0dccca9fd739f9ca5dee93c85bcf772941e81ac422c94847"}
Sep 05 08:12:03.408571 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.408555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6" event={"ID":"fcc5e8b1-a638-435f-b790-9f13fdc2a45a","Type":"ContainerDied","Data":"879757135d2815c47fe0e6cd5e2afc9f963734b5889792d11f5919ef018197ef"}
Sep 05 08:12:03.408617 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.408585    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="879757135d2815c47fe0e6cd5e2afc9f963734b5889792d11f5919ef018197ef"
Sep 05 08:12:03.425504 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.425478    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:12:03.486619 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.486586    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrksl\" (UniqueName: \"kubernetes.io/projected/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-kube-api-access-hrksl\") pod \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") "
Sep 05 08:12:03.486821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.486640    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-openshift-service-ca\") pod \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") "
Sep 05 08:12:03.486821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.486677    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-maas-api-tls\") pod \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") "
Sep 05 08:12:03.486821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.486708    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-metrics-certs\") pod \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\" (UID: \"fcc5e8b1-a638-435f-b790-9f13fdc2a45a\") "
Sep 05 08:12:03.487266 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.487236    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fcc5e8b1-a638-435f-b790-9f13fdc2a45a" (UID: "fcc5e8b1-a638-435f-b790-9f13fdc2a45a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:12:03.489265 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.489240    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fcc5e8b1-a638-435f-b790-9f13fdc2a45a" (UID: "fcc5e8b1-a638-435f-b790-9f13fdc2a45a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:12:03.489611 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.489581    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-kube-api-access-hrksl" (OuterVolumeSpecName: "kube-api-access-hrksl") pod "fcc5e8b1-a638-435f-b790-9f13fdc2a45a" (UID: "fcc5e8b1-a638-435f-b790-9f13fdc2a45a"). InnerVolumeSpecName "kube-api-access-hrksl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:12:03.489712 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.489664    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fcc5e8b1-a638-435f-b790-9f13fdc2a45a" (UID: "fcc5e8b1-a638-435f-b790-9f13fdc2a45a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:12:03.587850 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.587820    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrksl\" (UniqueName: \"kubernetes.io/projected/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-kube-api-access-hrksl\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:03.587850 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.587848    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:03.588060 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.587862    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:03.588060 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:03.587873    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcc5e8b1-a638-435f-b790-9f13fdc2a45a-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:04.414962 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:04.414929    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"
Sep 05 08:12:04.436131 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:04.436094    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"]
Sep 05 08:12:04.439820 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:04.439781    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e171e4-8596655564-pr5s6"]
Sep 05 08:12:05.571702 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:05.571670    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcc5e8b1-a638-435f-b790-9f13fdc2a45a" path="/var/lib/kubelet/pods/fcc5e8b1-a638-435f-b790-9f13fdc2a45a/volumes"
Sep 05 08:12:25.970228 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:25.970191    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"]
Sep 05 08:12:25.970733 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:25.970449    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z" podUID="e39a9c02-31a1-470f-a508-a002b62343e7" containerName="maas-api" containerID="cri-o://831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8" gracePeriod=30
Sep 05 08:12:26.231925 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.231860    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:12:26.322079 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.322048    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-metrics-certs\") pod \"e39a9c02-31a1-470f-a508-a002b62343e7\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") "
Sep 05 08:12:26.322280 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.322141    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtm4g\" (UniqueName: \"kubernetes.io/projected/e39a9c02-31a1-470f-a508-a002b62343e7-kube-api-access-gtm4g\") pod \"e39a9c02-31a1-470f-a508-a002b62343e7\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") "
Sep 05 08:12:26.322280 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.322198    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-maas-api-tls\") pod \"e39a9c02-31a1-470f-a508-a002b62343e7\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") "
Sep 05 08:12:26.322280 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.322237    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39a9c02-31a1-470f-a508-a002b62343e7-openshift-service-ca\") pod \"e39a9c02-31a1-470f-a508-a002b62343e7\" (UID: \"e39a9c02-31a1-470f-a508-a002b62343e7\") "
Sep 05 08:12:26.322950 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.322916    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e39a9c02-31a1-470f-a508-a002b62343e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e39a9c02-31a1-470f-a508-a002b62343e7" (UID: "e39a9c02-31a1-470f-a508-a002b62343e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:12:26.325148 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.325122    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e39a9c02-31a1-470f-a508-a002b62343e7" (UID: "e39a9c02-31a1-470f-a508-a002b62343e7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:12:26.325269 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.325233    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e39a9c02-31a1-470f-a508-a002b62343e7" (UID: "e39a9c02-31a1-470f-a508-a002b62343e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:12:26.325269 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.325251    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39a9c02-31a1-470f-a508-a002b62343e7-kube-api-access-gtm4g" (OuterVolumeSpecName: "kube-api-access-gtm4g") pod "e39a9c02-31a1-470f-a508-a002b62343e7" (UID: "e39a9c02-31a1-470f-a508-a002b62343e7"). InnerVolumeSpecName "kube-api-access-gtm4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:12:26.423916 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.423883    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:26.423916 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.423913    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtm4g\" (UniqueName: \"kubernetes.io/projected/e39a9c02-31a1-470f-a508-a002b62343e7-kube-api-access-gtm4g\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:26.423916 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.423923    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e39a9c02-31a1-470f-a508-a002b62343e7-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:26.424166 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.423932    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e39a9c02-31a1-470f-a508-a002b62343e7-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:26.543606 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.543494    2569 generic.go:356] "Generic (PLEG): container finished" podID="e39a9c02-31a1-470f-a508-a002b62343e7" containerID="831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8" exitCode=0
Sep 05 08:12:26.543606 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.543562    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"
Sep 05 08:12:26.543802 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.543561    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z" event={"ID":"e39a9c02-31a1-470f-a508-a002b62343e7","Type":"ContainerDied","Data":"831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8"}
Sep 05 08:12:26.543802 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.543697    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z" event={"ID":"e39a9c02-31a1-470f-a508-a002b62343e7","Type":"ContainerDied","Data":"180da7087b0ea73a64326758fb8d1d0ad3054f04bb7ca90bd8663dba7f1a653e"}
Sep 05 08:12:26.543802 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.543721    2569 scope.go:117] "RemoveContainer" containerID="831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8"
Sep 05 08:12:26.555510 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.555487    2569 scope.go:117] "RemoveContainer" containerID="831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8"
Sep 05 08:12:26.555873 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:12:26.555854    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8\": container with ID starting with 831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8 not found: ID does not exist" containerID="831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8"
Sep 05 08:12:26.555948 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.555883    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8"} err="failed to get container status \"831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8\": rpc error: code = NotFound desc = could not find container \"831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8\": container with ID starting with 831c1397cb0fa9f97df6c4576d66ad3559a572994dcc37515a7ec873fa6ca9f8 not found: ID does not exist"
Sep 05 08:12:26.569188 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.569160    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"]
Sep 05 08:12:26.571316 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:26.571293    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d850e3-7d799944c-tfk4z"]
Sep 05 08:12:27.572041 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:27.572008    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e39a9c02-31a1-470f-a508-a002b62343e7" path="/var/lib/kubelet/pods/e39a9c02-31a1-470f-a508-a002b62343e7/volumes"
Sep 05 08:12:50.488127 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:50.488087    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"]
Sep 05 08:12:50.488662 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:50.488369    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy" containerID="cri-o://96d0196611fced916637214ad47f205cadf36934f17cec7b168bc5830bb165fa" gracePeriod=30
Sep 05 08:12:55.715594 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.715563    2569 generic.go:356] "Generic (PLEG): container finished" podID="73a0dc85-80c0-4342-b877-e48469c83f12" containerID="96d0196611fced916637214ad47f205cadf36934f17cec7b168bc5830bb165fa" exitCode=0
Sep 05 08:12:55.716018 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.715652    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" event={"ID":"73a0dc85-80c0-4342-b877-e48469c83f12","Type":"ContainerDied","Data":"96d0196611fced916637214ad47f205cadf36934f17cec7b168bc5830bb165fa"}
Sep 05 08:12:55.716018 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.715715    2569 scope.go:117] "RemoveContainer" containerID="ccd13be5d8caccbabd4febb4599a851a4af9e2d55d52a46df68159f2035f9ec4"
Sep 05 08:12:55.753290 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.753224    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:12:55.836305 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836269    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a0dc85-80c0-4342-b877-e48469c83f12-istiod-ca-cert\") pod \"73a0dc85-80c0-4342-b877-e48469c83f12\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") "
Sep 05 08:12:55.836494 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836319    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4hpz\" (UniqueName: \"kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-kube-api-access-m4hpz\") pod \"73a0dc85-80c0-4342-b877-e48469c83f12\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") "
Sep 05 08:12:55.836494 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836370    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-certs\") pod \"73a0dc85-80c0-4342-b877-e48469c83f12\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") "
Sep 05 08:12:55.836494 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836423    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-socket\") pod \"73a0dc85-80c0-4342-b877-e48469c83f12\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") "
Sep 05 08:12:55.836494 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836457    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-istio-token\") pod \"73a0dc85-80c0-4342-b877-e48469c83f12\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") "
Sep 05 08:12:55.836761 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836515    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-envoy\") pod \"73a0dc85-80c0-4342-b877-e48469c83f12\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") "
Sep 05 08:12:55.836761 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836610    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a0dc85-80c0-4342-b877-e48469c83f12-istio-podinfo\") pod \"73a0dc85-80c0-4342-b877-e48469c83f12\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") "
Sep 05 08:12:55.836761 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836638    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-data\") pod \"73a0dc85-80c0-4342-b877-e48469c83f12\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") "
Sep 05 08:12:55.836761 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836668    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-credential-socket\") pod \"73a0dc85-80c0-4342-b877-e48469c83f12\" (UID: \"73a0dc85-80c0-4342-b877-e48469c83f12\") "
Sep 05 08:12:55.836968 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836754    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73a0dc85-80c0-4342-b877-e48469c83f12" (UID: "73a0dc85-80c0-4342-b877-e48469c83f12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:55.836968 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836781    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a0dc85-80c0-4342-b877-e48469c83f12-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73a0dc85-80c0-4342-b877-e48469c83f12" (UID: "73a0dc85-80c0-4342-b877-e48469c83f12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:12:55.836968 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.836889    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73a0dc85-80c0-4342-b877-e48469c83f12" (UID: "73a0dc85-80c0-4342-b877-e48469c83f12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:55.837122 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.837001    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-data" (OuterVolumeSpecName: "istio-data") pod "73a0dc85-80c0-4342-b877-e48469c83f12" (UID: "73a0dc85-80c0-4342-b877-e48469c83f12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:55.837122 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.837021    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73a0dc85-80c0-4342-b877-e48469c83f12" (UID: "73a0dc85-80c0-4342-b877-e48469c83f12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:55.837269 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.837253    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:55.837331 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.837270    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:55.837331 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.837281    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:55.837331 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.837295    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:55.837331 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.837310    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a0dc85-80c0-4342-b877-e48469c83f12-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:55.839055 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.839023    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73a0dc85-80c0-4342-b877-e48469c83f12-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73a0dc85-80c0-4342-b877-e48469c83f12" (UID: "73a0dc85-80c0-4342-b877-e48469c83f12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:12:55.839055 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.839043    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-istio-token" (OuterVolumeSpecName: "istio-token") pod "73a0dc85-80c0-4342-b877-e48469c83f12" (UID: "73a0dc85-80c0-4342-b877-e48469c83f12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:12:55.839196 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.839053    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73a0dc85-80c0-4342-b877-e48469c83f12" (UID: "73a0dc85-80c0-4342-b877-e48469c83f12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:12:55.839196 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.839118    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-kube-api-access-m4hpz" (OuterVolumeSpecName: "kube-api-access-m4hpz") pod "73a0dc85-80c0-4342-b877-e48469c83f12" (UID: "73a0dc85-80c0-4342-b877-e48469c83f12"). InnerVolumeSpecName "kube-api-access-m4hpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:12:55.938897 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.938861    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a0dc85-80c0-4342-b877-e48469c83f12-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:55.938897 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.938892    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4hpz\" (UniqueName: \"kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-kube-api-access-m4hpz\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:55.938897 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.938901    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a0dc85-80c0-4342-b877-e48469c83f12-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:55.938897 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:55.938910    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a0dc85-80c0-4342-b877-e48469c83f12-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:12:56.722776 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:56.722751    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"
Sep 05 08:12:56.723238 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:56.722789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t" event={"ID":"73a0dc85-80c0-4342-b877-e48469c83f12","Type":"ContainerDied","Data":"6d8c8428170b4e51f00c64f945e5c695c2eee32f1786d698d877914876b62185"}
Sep 05 08:12:56.723238 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:56.722849    2569 scope.go:117] "RemoveContainer" containerID="96d0196611fced916637214ad47f205cadf36934f17cec7b168bc5830bb165fa"
Sep 05 08:12:56.746094 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:56.746068    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"]
Sep 05 08:12:56.750031 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:56.750009    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8b6928-openshift-default-7d8b466968-k4h4t"]
Sep 05 08:12:57.571068 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:12:57.571033    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" path="/var/lib/kubelet/pods/73a0dc85-80c0-4342-b877-e48469c83f12/volumes"
Sep 05 08:13:00.915881 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:00.915829    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"]
Sep 05 08:13:00.916461 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:00.916109    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5" podUID="da864b70-d5a4-4394-91ca-8de0964852b1" containerName="maas-api" containerID="cri-o://940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580" gracePeriod=30
Sep 05 08:13:01.174406 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.174347    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:13:01.294815 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.294787    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da864b70-d5a4-4394-91ca-8de0964852b1-openshift-service-ca\") pod \"da864b70-d5a4-4394-91ca-8de0964852b1\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") "
Sep 05 08:13:01.295000 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.294822    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-maas-api-tls\") pod \"da864b70-d5a4-4394-91ca-8de0964852b1\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") "
Sep 05 08:13:01.295000 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.294897    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-metrics-certs\") pod \"da864b70-d5a4-4394-91ca-8de0964852b1\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") "
Sep 05 08:13:01.295000 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.294927    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9rxx\" (UniqueName: \"kubernetes.io/projected/da864b70-d5a4-4394-91ca-8de0964852b1-kube-api-access-g9rxx\") pod \"da864b70-d5a4-4394-91ca-8de0964852b1\" (UID: \"da864b70-d5a4-4394-91ca-8de0964852b1\") "
Sep 05 08:13:01.295291 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.295243    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da864b70-d5a4-4394-91ca-8de0964852b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da864b70-d5a4-4394-91ca-8de0964852b1" (UID: "da864b70-d5a4-4394-91ca-8de0964852b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:13:01.297262 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.297233    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da864b70-d5a4-4394-91ca-8de0964852b1" (UID: "da864b70-d5a4-4394-91ca-8de0964852b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:13:01.297390 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.297345    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da864b70-d5a4-4394-91ca-8de0964852b1-kube-api-access-g9rxx" (OuterVolumeSpecName: "kube-api-access-g9rxx") pod "da864b70-d5a4-4394-91ca-8de0964852b1" (UID: "da864b70-d5a4-4394-91ca-8de0964852b1"). InnerVolumeSpecName "kube-api-access-g9rxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:13:01.298031 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.298001    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "da864b70-d5a4-4394-91ca-8de0964852b1" (UID: "da864b70-d5a4-4394-91ca-8de0964852b1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:13:01.396185 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.396146    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:01.396185 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.396181    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9rxx\" (UniqueName: \"kubernetes.io/projected/da864b70-d5a4-4394-91ca-8de0964852b1-kube-api-access-g9rxx\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:01.396185 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.396191    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da864b70-d5a4-4394-91ca-8de0964852b1-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:01.396433 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.396200    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da864b70-d5a4-4394-91ca-8de0964852b1-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:01.751952 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.751921    2569 generic.go:356] "Generic (PLEG): container finished" podID="da864b70-d5a4-4394-91ca-8de0964852b1" containerID="940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580" exitCode=0
Sep 05 08:13:01.752149 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.751968    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"
Sep 05 08:13:01.752149 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.752008    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5" event={"ID":"da864b70-d5a4-4394-91ca-8de0964852b1","Type":"ContainerDied","Data":"940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580"}
Sep 05 08:13:01.752149 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.752070    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5" event={"ID":"da864b70-d5a4-4394-91ca-8de0964852b1","Type":"ContainerDied","Data":"21f9a215733d2a7cf5a5cfa35317e2dce948c3f3549bac53fd255f0989a99998"}
Sep 05 08:13:01.752149 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.752100    2569 scope.go:117] "RemoveContainer" containerID="940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580"
Sep 05 08:13:01.761153 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.761137    2569 scope.go:117] "RemoveContainer" containerID="940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580"
Sep 05 08:13:01.761458 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:13:01.761439    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580\": container with ID starting with 940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580 not found: ID does not exist" containerID="940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580"
Sep 05 08:13:01.761633 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.761464    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580"} err="failed to get container status \"940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580\": rpc error: code = NotFound desc = could not find container \"940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580\": container with ID starting with 940ef81c60d719b899e439232e04592183be4e3ed1e911ebc17275fbec650580 not found: ID does not exist"
Sep 05 08:13:01.772550 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.772501    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"]
Sep 05 08:13:01.776103 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:01.776072    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cec446-75bdf79648-cknx5"]
Sep 05 08:13:03.572081 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:03.572047    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da864b70-d5a4-4394-91ca-8de0964852b1" path="/var/lib/kubelet/pods/da864b70-d5a4-4394-91ca-8de0964852b1/volumes"
Sep 05 08:13:07.313389 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:07.313357    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"]
Sep 05 08:13:07.313903 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:07.313595    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy" containerID="cri-o://9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390" gracePeriod=30
Sep 05 08:13:12.356794 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:13:12.356759    2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8c9e09_cd2b_497d_b0bc_33067258136c.slice/crio-conmon-9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8c9e09_cd2b_497d_b0bc_33067258136c.slice/crio-9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390.scope\": RecentStats: unable to find data in memory cache]"
Sep 05 08:13:12.572331 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.572306    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:13:12.608677 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.608590    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-podinfo\") pod \"0a8c9e09-cd2b-497d-b0bc-33067258136c\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") "
Sep 05 08:13:12.608677 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.608642    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-socket\") pod \"0a8c9e09-cd2b-497d-b0bc-33067258136c\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") "
Sep 05 08:13:12.608677 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.608664    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-data\") pod \"0a8c9e09-cd2b-497d-b0bc-33067258136c\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") "
Sep 05 08:13:12.608975 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.608786    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cgkd\" (UniqueName: \"kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-kube-api-access-5cgkd\") pod \"0a8c9e09-cd2b-497d-b0bc-33067258136c\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") "
Sep 05 08:13:12.608975 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.608889    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-envoy\") pod \"0a8c9e09-cd2b-497d-b0bc-33067258136c\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") "
Sep 05 08:13:12.608975 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.608924    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-token\") pod \"0a8c9e09-cd2b-497d-b0bc-33067258136c\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") "
Sep 05 08:13:12.608975 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.608950    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a8c9e09-cd2b-497d-b0bc-33067258136c" (UID: "0a8c9e09-cd2b-497d-b0bc-33067258136c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:12.608975 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.608957    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a8c9e09-cd2b-497d-b0bc-33067258136c" (UID: "0a8c9e09-cd2b-497d-b0bc-33067258136c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:12.609249 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.609019    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-credential-socket\") pod \"0a8c9e09-cd2b-497d-b0bc-33067258136c\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") "
Sep 05 08:13:12.609249 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.609052    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a8c9e09-cd2b-497d-b0bc-33067258136c-istiod-ca-cert\") pod \"0a8c9e09-cd2b-497d-b0bc-33067258136c\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") "
Sep 05 08:13:12.609249 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.609133    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-certs\") pod \"0a8c9e09-cd2b-497d-b0bc-33067258136c\" (UID: \"0a8c9e09-cd2b-497d-b0bc-33067258136c\") "
Sep 05 08:13:12.609405 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.609264    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a8c9e09-cd2b-497d-b0bc-33067258136c" (UID: "0a8c9e09-cd2b-497d-b0bc-33067258136c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:12.609556 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.609512    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a8c9e09-cd2b-497d-b0bc-33067258136c" (UID: "0a8c9e09-cd2b-497d-b0bc-33067258136c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:12.610012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.609786    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a8c9e09-cd2b-497d-b0bc-33067258136c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a8c9e09-cd2b-497d-b0bc-33067258136c" (UID: "0a8c9e09-cd2b-497d-b0bc-33067258136c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:13:12.610012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.609774    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:12.610012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.609862    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:12.610012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.609879    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:12.610012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.609894    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:12.611245 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.611223    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a8c9e09-cd2b-497d-b0bc-33067258136c" (UID: "0a8c9e09-cd2b-497d-b0bc-33067258136c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:13:12.611401 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.611384    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a8c9e09-cd2b-497d-b0bc-33067258136c" (UID: "0a8c9e09-cd2b-497d-b0bc-33067258136c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:13:12.611716 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.611700    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-kube-api-access-5cgkd" (OuterVolumeSpecName: "kube-api-access-5cgkd") pod "0a8c9e09-cd2b-497d-b0bc-33067258136c" (UID: "0a8c9e09-cd2b-497d-b0bc-33067258136c"). InnerVolumeSpecName "kube-api-access-5cgkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:13:12.611815 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.611792    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a8c9e09-cd2b-497d-b0bc-33067258136c" (UID: "0a8c9e09-cd2b-497d-b0bc-33067258136c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:13:12.710864 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.710831    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:12.710864 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.710857    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cgkd\" (UniqueName: \"kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-kube-api-access-5cgkd\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:12.710864 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.710868    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:12.711113 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.710877    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a8c9e09-cd2b-497d-b0bc-33067258136c-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:12.711113 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.710885    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a8c9e09-cd2b-497d-b0bc-33067258136c-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:13:12.817041 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.817006    2569 generic.go:356] "Generic (PLEG): container finished" podID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerID="9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390" exitCode=0
Sep 05 08:13:12.817217 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.817058    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"
Sep 05 08:13:12.817217 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.817095    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" event={"ID":"0a8c9e09-cd2b-497d-b0bc-33067258136c","Type":"ContainerDied","Data":"9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390"}
Sep 05 08:13:12.817217 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.817154    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr" event={"ID":"0a8c9e09-cd2b-497d-b0bc-33067258136c","Type":"ContainerDied","Data":"812f1649439ea0682aab65c40d003d054ebd83e912df3038abb23e9e80822636"}
Sep 05 08:13:12.817217 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.817185    2569 scope.go:117] "RemoveContainer" containerID="9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390"
Sep 05 08:13:12.827230 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.827212    2569 scope.go:117] "RemoveContainer" containerID="7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c"
Sep 05 08:13:12.835546 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.835501    2569 scope.go:117] "RemoveContainer" containerID="9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390"
Sep 05 08:13:12.835889 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:13:12.835850    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390\": container with ID starting with 9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390 not found: ID does not exist" containerID="9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390"
Sep 05 08:13:12.835979 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.835892    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390"} err="failed to get container status \"9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390\": rpc error: code = NotFound desc = could not find container \"9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390\": container with ID starting with 9a2ae3eeb758e980dff3ababa2b74daab2d1ca4f65b4b317efd3e7135f457390 not found: ID does not exist"
Sep 05 08:13:12.835979 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.835916    2569 scope.go:117] "RemoveContainer" containerID="7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c"
Sep 05 08:13:12.836481 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:13:12.836304    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c\": container with ID starting with 7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c not found: ID does not exist" containerID="7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c"
Sep 05 08:13:12.836481 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.836335    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c"} err="failed to get container status \"7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c\": rpc error: code = NotFound desc = could not find container \"7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c\": container with ID starting with 7cbb577502c4a7bcf561eb917c35f5701151eab269efae06f502e692ce4b2f9c not found: ID does not exist"
Sep 05 08:13:12.839152 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.839133    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"]
Sep 05 08:13:12.843400 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:12.843379    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-cec446-openshift-default-5dd5bb76f8-262pr"]
Sep 05 08:13:13.570707 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:13.570672    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" path="/var/lib/kubelet/pods/0a8c9e09-cd2b-497d-b0bc-33067258136c/volumes"
Sep 05 08:13:20.066381 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066344    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"]
Sep 05 08:13:20.066853 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066839    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy"
Sep 05 08:13:20.066907 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066855    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy"
Sep 05 08:13:20.066907 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066864    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39a9c02-31a1-470f-a508-a002b62343e7" containerName="maas-api"
Sep 05 08:13:20.066907 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066869    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39a9c02-31a1-470f-a508-a002b62343e7" containerName="maas-api"
Sep 05 08:13:20.066907 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066879    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcc5e8b1-a638-435f-b790-9f13fdc2a45a" containerName="maas-api"
Sep 05 08:13:20.066907 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066885    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcc5e8b1-a638-435f-b790-9f13fdc2a45a" containerName="maas-api"
Sep 05 08:13:20.066907 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066903    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da864b70-d5a4-4394-91ca-8de0964852b1" containerName="maas-api"
Sep 05 08:13:20.066907 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066909    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="da864b70-d5a4-4394-91ca-8de0964852b1" containerName="maas-api"
Sep 05 08:13:20.067138 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066929    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy"
Sep 05 08:13:20.067138 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066935    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy"
Sep 05 08:13:20.067138 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066943    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy"
Sep 05 08:13:20.067138 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.066948    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy"
Sep 05 08:13:20.067138 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.067020    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy"
Sep 05 08:13:20.067138 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.067029    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy"
Sep 05 08:13:20.067138 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.067035    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39a9c02-31a1-470f-a508-a002b62343e7" containerName="maas-api"
Sep 05 08:13:20.067138 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.067043    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="da864b70-d5a4-4394-91ca-8de0964852b1" containerName="maas-api"
Sep 05 08:13:20.067138 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.067052    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcc5e8b1-a638-435f-b790-9f13fdc2a45a" containerName="maas-api"
Sep 05 08:13:20.071551 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.071505    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.074251 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.074232    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-3d4dc1-openshift-default-dockercfg-bbsmk\""
Sep 05 08:13:20.081014 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.080991    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"]
Sep 05 08:13:20.188455 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.188414    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9c9c\" (UniqueName: \"kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-kube-api-access-j9c9c\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.188455 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.188457    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-envoy\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.188697 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.188563    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-data\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.188697 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.188605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7553196-e42d-403c-a14d-d9030613aa1f-istiod-ca-cert\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.188697 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.188636    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7553196-e42d-403c-a14d-d9030613aa1f-istio-podinfo\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.188697 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.188666    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-socket\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.188838 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.188712    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-credential-socket\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.188838 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.188743    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-istio-token\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.188838 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.188785    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-certs\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290290 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290239    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7553196-e42d-403c-a14d-d9030613aa1f-istio-podinfo\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290290 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290297    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-socket\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290506 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290315    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-credential-socket\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290506 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290340    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-istio-token\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290506 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290381    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-certs\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290506 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290441    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9c9c\" (UniqueName: \"kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-kube-api-access-j9c9c\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290506 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290468    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-envoy\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290506 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290503    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-data\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290891 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290521    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7553196-e42d-403c-a14d-d9030613aa1f-istiod-ca-cert\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290891 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290751    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-socket\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290891 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290770    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-credential-socket\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.290891 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.290831    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-certs\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.291371 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.291342    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7553196-e42d-403c-a14d-d9030613aa1f-istiod-ca-cert\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.291483 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.291376    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-data\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.293052 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.293032    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7553196-e42d-403c-a14d-d9030613aa1f-istio-podinfo\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.293293 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.293275    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-envoy\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.298666 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.298647    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-istio-token\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.298747 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.298730    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9c9c\" (UniqueName: \"kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-kube-api-access-j9c9c\") pod \"e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") " pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.383602 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.383567    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:20.659940 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.659717    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"]
Sep 05 08:13:20.662689 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:13:20.662657    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7553196_e42d_403c_a14d_d9030613aa1f.slice/crio-0601f2cc4737d108056c67d67a72f9ed991eceb2dd091acd83b7a89f7fb132d9 WatchSource:0}: Error finding container 0601f2cc4737d108056c67d67a72f9ed991eceb2dd091acd83b7a89f7fb132d9: Status 404 returned error can't find the container with id 0601f2cc4737d108056c67d67a72f9ed991eceb2dd091acd83b7a89f7fb132d9
Sep 05 08:13:20.664846 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.664809    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:13:20.664937 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.664880    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:13:20.664937 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.664907    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:13:20.864469 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.864438    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" event={"ID":"e7553196-e42d-403c-a14d-d9030613aa1f","Type":"ContainerStarted","Data":"3b9d77dd8f53d65f662e79f0945663e0d7d2ed65fa4bdb75ed57a96628733d65"}
Sep 05 08:13:20.864691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.864486    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" event={"ID":"e7553196-e42d-403c-a14d-d9030613aa1f","Type":"ContainerStarted","Data":"0601f2cc4737d108056c67d67a72f9ed991eceb2dd091acd83b7a89f7fb132d9"}
Sep 05 08:13:20.884628 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:20.884575    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" podStartSLOduration=0.884560037 podStartE2EDuration="884.560037ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:13:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:13:20.883123901 +0000 UTC m=+1481.828445698" watchObservedRunningTime="2026-09-05 08:13:20.884560037 +0000 UTC m=+1481.829881841"
Sep 05 08:13:21.384151 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:21.384110    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:21.389620 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:21.389596    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:21.870741 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:21.870713    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:21.871623 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:21.871604    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:37.913681 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:37.913635    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 08:13:37.914173 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:37.913706    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 08:13:38.966576 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:38.966517    2569 generic.go:356] "Generic (PLEG): container finished" podID="e7553196-e42d-403c-a14d-d9030613aa1f" containerID="3b9d77dd8f53d65f662e79f0945663e0d7d2ed65fa4bdb75ed57a96628733d65" exitCode=0
Sep 05 08:13:38.966576 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:38.966558    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" event={"ID":"e7553196-e42d-403c-a14d-d9030613aa1f","Type":"ContainerDied","Data":"3b9d77dd8f53d65f662e79f0945663e0d7d2ed65fa4bdb75ed57a96628733d65"}
Sep 05 08:13:38.967008 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:38.966988    2569 scope.go:117] "RemoveContainer" containerID="3b9d77dd8f53d65f662e79f0945663e0d7d2ed65fa4bdb75ed57a96628733d65"
Sep 05 08:13:38.968159 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:38.968129    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:13:38.968222 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:38.968201    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:13:38.968271 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:38.968232    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:13:39.670821 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:39.670791    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:13:39.675993 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:39.675973    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:13:39.974924 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:39.974835    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" event={"ID":"e7553196-e42d-403c-a14d-d9030613aa1f","Type":"ContainerStarted","Data":"6f58da5f5d605afd2cc07ad9c7d9d25b5e4f59407a0046b6f22ebaf1eaa1996f"}
Sep 05 08:13:40.384550 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:40.384492    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:40.384793 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:40.384773    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 05 08:13:40.384840 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:40.384822    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 05 08:13:41.384889 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:41.384855    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 05 08:13:41.385270 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:41.384910    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 05 08:13:42.384077 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:42.384044    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 05 08:13:42.384301 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:42.384097    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 05 08:13:43.384483 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:43.384445    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 05 08:13:43.384878 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:43.384511    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 05 08:13:44.384627 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:44.384589    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 05 08:13:44.385027 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:44.384663    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 05 08:13:45.413466 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:45.413429    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:45.413965 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:45.413942    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:45.414812 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:45.414790    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:13:48.877205 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.877174    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"]
Sep 05 08:13:48.877877 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.877854    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy"
Sep 05 08:13:48.877977 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.877880    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy"
Sep 05 08:13:48.878057 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.878043    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a0dc85-80c0-4342-b877-e48469c83f12" containerName="istio-proxy"
Sep 05 08:13:48.878112 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.878069    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a8c9e09-cd2b-497d-b0bc-33067258136c" containerName="istio-proxy"
Sep 05 08:13:48.882754 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.882731    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:48.885820 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.885793    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-d14aea-openshift-default-dockercfg-4cfgt\""
Sep 05 08:13:48.893898 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.893786    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"]
Sep 05 08:13:48.929816 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.929784    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25866f6f-8128-42d2-b896-24e5e778a6d0-istiod-ca-cert\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:48.930010 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.929844    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-certs\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:48.930010 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.929935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-socket\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:48.930010 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.929965    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-podinfo\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:48.930142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.930060    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-data\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:48.930142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.930086    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-credential-socket\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:48.930142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.930117    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnnl7\" (UniqueName: \"kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-kube-api-access-wnnl7\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:48.930238 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.930141    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-token\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:48.930238 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:48.930167    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-envoy\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.030902 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.030869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-socket\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.030902 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.030901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-podinfo\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031145 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.030956    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-data\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031145 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.030982    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-credential-socket\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031145 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.031003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnnl7\" (UniqueName: \"kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-kube-api-access-wnnl7\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031145 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.031031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-token\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031145 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.031062    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-envoy\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031145 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.031107    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25866f6f-8128-42d2-b896-24e5e778a6d0-istiod-ca-cert\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031472 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.031154    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-certs\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031472 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.031346    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-socket\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031472 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.031359    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-data\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031472 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.031437    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-credential-socket\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031703 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.031558    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-certs\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.031970 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.031954    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25866f6f-8128-42d2-b896-24e5e778a6d0-istiod-ca-cert\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.033673 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.033645    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-envoy\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.033978 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.033953    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-podinfo\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.040082 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.040054    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-token\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.040208 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.040186    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnnl7\" (UniqueName: \"kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-kube-api-access-wnnl7\") pod \"e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") " pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.195672 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.195586    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:49.479568 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.478646    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"]
Sep 05 08:13:49.485807 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.485774    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:13:49.485917 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.485840    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:13:49.485917 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:49.485869    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:13:50.029565 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:50.029518    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" event={"ID":"25866f6f-8128-42d2-b896-24e5e778a6d0","Type":"ContainerStarted","Data":"0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353"}
Sep 05 08:13:50.029995 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:50.029587    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" event={"ID":"25866f6f-8128-42d2-b896-24e5e778a6d0","Type":"ContainerStarted","Data":"cf6437c16928e262914dfaf41bdfe8bec99d083a7aaa09caec67b682bada5ee5"}
Sep 05 08:13:50.050208 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:50.050155    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" podStartSLOduration=2.050133885 podStartE2EDuration="2.050133885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:13:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:13:50.048494611 +0000 UTC m=+1510.993816417" watchObservedRunningTime="2026-09-05 08:13:50.050133885 +0000 UTC m=+1510.995455693"
Sep 05 08:13:50.195794 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:50.195760    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:50.201367 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:50.201343    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:51.035797 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:51.035765    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:51.036785 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:51.036766    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:13:53.089415 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.089373    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"]
Sep 05 08:13:53.093959 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.093935    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.097012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.096984    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-d14aea-serving-cert\""
Sep 05 08:13:53.102979 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.102958    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"]
Sep 05 08:13:53.178932 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.178900    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-metrics-certs\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.179085 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.179004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.179142 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.179127    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkc5l\" (UniqueName: \"kubernetes.io/projected/8f61bbde-f924-475c-9854-98de189e7ef3-kube-api-access-tkc5l\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.179242 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.179230    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f61bbde-f924-475c-9854-98de189e7ef3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.280406 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.280369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkc5l\" (UniqueName: \"kubernetes.io/projected/8f61bbde-f924-475c-9854-98de189e7ef3-kube-api-access-tkc5l\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.280618 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.280451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f61bbde-f924-475c-9854-98de189e7ef3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.280618 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.280517    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-metrics-certs\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.280618 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.280575    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.280739 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:13:53.280700    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-serving-cert: secret "maas-api-e2e-ipp-b-d14aea-serving-cert" not found
Sep 05 08:13:53.280774 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:13:53.280766    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-maas-api-tls podName:8f61bbde-f924-475c-9854-98de189e7ef3 nodeName:}" failed. No retries permitted until 2026-09-05 08:13:53.780745613 +0000 UTC m=+1514.726067415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-maas-api-tls") pod "maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5" (UID: "8f61bbde-f924-475c-9854-98de189e7ef3") : secret "maas-api-e2e-ipp-b-d14aea-serving-cert" not found
Sep 05 08:13:53.281139 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.281106    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f61bbde-f924-475c-9854-98de189e7ef3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.283169 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.283148    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-metrics-certs\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.291495 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.291468    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkc5l\" (UniqueName: \"kubernetes.io/projected/8f61bbde-f924-475c-9854-98de189e7ef3-kube-api-access-tkc5l\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.786620 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.786585    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:53.789176 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:53.789147    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:54.008156 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:54.008119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:54.285683 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:54.285653    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"]
Sep 05 08:13:54.286723 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:13:54.286690    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f61bbde_f924_475c_9854_98de189e7ef3.slice/crio-bc837e26cec2f0d1974f9fefcaf9cc9e635dbcbe96de4852922649ea4f8f9579 WatchSource:0}: Error finding container bc837e26cec2f0d1974f9fefcaf9cc9e635dbcbe96de4852922649ea4f8f9579: Status 404 returned error can't find the container with id bc837e26cec2f0d1974f9fefcaf9cc9e635dbcbe96de4852922649ea4f8f9579
Sep 05 08:13:54.288273 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:54.288254    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 08:13:55.060599 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:55.060467    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5" event={"ID":"8f61bbde-f924-475c-9854-98de189e7ef3","Type":"ContainerStarted","Data":"338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4"}
Sep 05 08:13:55.060599 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:55.060571    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5" event={"ID":"8f61bbde-f924-475c-9854-98de189e7ef3","Type":"ContainerStarted","Data":"bc837e26cec2f0d1974f9fefcaf9cc9e635dbcbe96de4852922649ea4f8f9579"}
Sep 05 08:13:55.060840 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:55.060604    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:13:55.079380 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:13:55.079321    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5" podStartSLOduration=1.7146392339999998 podStartE2EDuration="2.079308118s" totalImagesPullingTime="364.668884ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:13:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:13:55.076896787 +0000 UTC m=+1516.022218592" watchObservedRunningTime="2026-09-05 08:13:55.079308118 +0000 UTC m=+1516.024629923"
Sep 05 08:14:01.071383 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:01.071292    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:14:07.077748 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:07.077698    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 08:14:07.078344 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:07.077787    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 08:14:08.134596 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:08.134492    2569 generic.go:356] "Generic (PLEG): container finished" podID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerID="0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353" exitCode=0
Sep 05 08:14:08.134596 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:08.134567    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" event={"ID":"25866f6f-8128-42d2-b896-24e5e778a6d0","Type":"ContainerDied","Data":"0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353"}
Sep 05 08:14:08.135052 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:08.135010    2569 scope.go:117] "RemoveContainer" containerID="0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353"
Sep 05 08:14:08.136346 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:08.136320    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:14:08.136408 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:08.136388    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:14:08.136450 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:08.136416    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:14:09.146509 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:09.146465    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" event={"ID":"25866f6f-8128-42d2-b896-24e5e778a6d0","Type":"ContainerStarted","Data":"a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af"}
Sep 05 08:14:09.195898 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:09.195860    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:14:09.196107 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:09.196086    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 08:14:09.196193 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:09.196138    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 08:14:10.196951 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:10.196916    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 08:14:10.197345 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:10.196980    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 08:14:11.196283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:11.196240    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 08:14:11.196461 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:11.196311    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 08:14:12.195984 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:12.195945    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 08:14:12.196400 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:12.195999    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 08:14:13.196455 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:13.196423    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 08:14:13.196879 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:13.196479    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 08:14:14.196084 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:14.196046    2569 patch_prober.go:28] interesting pod/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 08:14:14.196314 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:14.196109    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 08:14:15.200515 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:15.200488    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:14:15.200983 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:15.200785    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:14:15.201423 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:15.201406    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:14:22.241247 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:14:22.241212    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-fxvvn_31f840d8-c593-439e-9144-d2004574c575/payload-processing/0.log"
Sep 05 08:15:02.900509 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:02.900476    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-fxvvn_31f840d8-c593-439e-9144-d2004574c575/payload-processing/0.log"
Sep 05 08:15:03.596805 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.596766    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"]
Sep 05 08:15:03.600860 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.600842    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.603741 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.603719    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-d9d46f-openshift-default-dockercfg-7sm5x\""
Sep 05 08:15:03.611541 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.611501    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"]
Sep 05 08:15:03.714445 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.714395    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-data\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.714692 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.714604    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-credential-socket\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.714692 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.714651    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-envoy\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.714816 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.714724    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8498e642-890c-419a-bb25-50fe4803fa13-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.714883 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.714816    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8498e642-890c-419a-bb25-50fe4803fa13-istio-podinfo\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.714944 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.714882    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-socket\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.714944 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.714920    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmphq\" (UniqueName: \"kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-kube-api-access-dmphq\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.715029 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.715000    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-certs\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.715136 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.715056    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-istio-token\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.816404 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816370    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8498e642-890c-419a-bb25-50fe4803fa13-istio-podinfo\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.816630 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816416    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-socket\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.816630 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816433    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmphq\" (UniqueName: \"kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-kube-api-access-dmphq\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.816630 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816473    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-certs\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.816826 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816655    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-istio-token\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.816826 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-data\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.816826 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816796    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-credential-socket\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.816826 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816826    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-envoy\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.817057 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816830    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-socket\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.817057 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816876    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-certs\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.817057 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.816905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8498e642-890c-419a-bb25-50fe4803fa13-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.817057 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.817043    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-data\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.817369 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.817203    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-credential-socket\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.817569 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.817551    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8498e642-890c-419a-bb25-50fe4803fa13-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.819097 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.819075    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-envoy\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.819335 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.819314    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8498e642-890c-419a-bb25-50fe4803fa13-istio-podinfo\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.827784 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.827756    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-istio-token\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.828015 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.827995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmphq\" (UniqueName: \"kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-kube-api-access-dmphq\") pod \"e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") " pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:03.913570 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:03.913458    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:04.189864 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:04.189833    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"]
Sep 05 08:15:04.191125 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:15:04.191096    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8498e642_890c_419a_bb25_50fe4803fa13.slice/crio-5423a6ed68262c4dc7b8395c4aca2b1468b4530b6d7ea270d7821b9dc7477dee WatchSource:0}: Error finding container 5423a6ed68262c4dc7b8395c4aca2b1468b4530b6d7ea270d7821b9dc7477dee: Status 404 returned error can't find the container with id 5423a6ed68262c4dc7b8395c4aca2b1468b4530b6d7ea270d7821b9dc7477dee
Sep 05 08:15:04.193501 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:04.193470    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:15:04.193633 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:04.193556    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:15:04.193633 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:04.193592    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:15:04.476648 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:04.476553    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" event={"ID":"8498e642-890c-419a-bb25-50fe4803fa13","Type":"ContainerStarted","Data":"764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b"}
Sep 05 08:15:04.476648 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:04.476609    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" event={"ID":"8498e642-890c-419a-bb25-50fe4803fa13","Type":"ContainerStarted","Data":"5423a6ed68262c4dc7b8395c4aca2b1468b4530b6d7ea270d7821b9dc7477dee"}
Sep 05 08:15:04.500497 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:04.500444    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" podStartSLOduration=1.5004272950000002 podStartE2EDuration="1.500427295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:15:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:15:04.498199738 +0000 UTC m=+1585.443521543" watchObservedRunningTime="2026-09-05 08:15:04.500427295 +0000 UTC m=+1585.445749100"
Sep 05 08:15:04.913597 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:04.913564    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:05.920018 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:05.919983    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:06.487172 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:06.487142    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:06.488303 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:06.488282    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:25.610657 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:25.610621    2569 generic.go:356] "Generic (PLEG): container finished" podID="8498e642-890c-419a-bb25-50fe4803fa13" containerID="764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b" exitCode=0
Sep 05 08:15:25.611064 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:25.610695    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" event={"ID":"8498e642-890c-419a-bb25-50fe4803fa13","Type":"ContainerDied","Data":"764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b"}
Sep 05 08:15:25.611185 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:25.611169    2569 scope.go:117] "RemoveContainer" containerID="764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b"
Sep 05 08:15:25.612304 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:25.612280    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:15:25.612369 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:25.612357    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:15:25.612404 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:25.612391    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 05 08:15:26.621595 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:26.621560    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" event={"ID":"8498e642-890c-419a-bb25-50fe4803fa13","Type":"ContainerStarted","Data":"1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d"}
Sep 05 08:15:26.914698 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:26.914604    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:26.914846 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:26.914829    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 05 08:15:26.914891 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:26.914870    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 05 08:15:27.914499 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:27.914461    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 05 08:15:27.914943 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:27.914516    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 05 08:15:28.914343 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:28.914309    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 05 08:15:28.914514 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:28.914378    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 05 08:15:29.914090 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:29.914052    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 05 08:15:29.914490 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:29.914107    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 05 08:15:30.914951 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:30.914867    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 05 08:15:30.915310 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:30.914947    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 05 08:15:31.942915 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:31.942879    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:31.943440 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:31.943208    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:31.944101 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:31.944081    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:44.866742 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:44.862958    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"]
Sep 05 08:15:44.866742 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:44.863240    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy" containerID="cri-o://1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d" gracePeriod=30
Sep 05 08:15:46.945477 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:46.945439    2569 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 05 08:15:46.945877 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:46.945521    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 05 08:15:50.133899 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.133839    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:50.230935 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.230876    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-socket\") pod \"8498e642-890c-419a-bb25-50fe4803fa13\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") "
Sep 05 08:15:50.230935 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.230938    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8498e642-890c-419a-bb25-50fe4803fa13-istiod-ca-cert\") pod \"8498e642-890c-419a-bb25-50fe4803fa13\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") "
Sep 05 08:15:50.231179 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.230961    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-certs\") pod \"8498e642-890c-419a-bb25-50fe4803fa13\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") "
Sep 05 08:15:50.231179 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.230994    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8498e642-890c-419a-bb25-50fe4803fa13-istio-podinfo\") pod \"8498e642-890c-419a-bb25-50fe4803fa13\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") "
Sep 05 08:15:50.231179 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231021    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-credential-socket\") pod \"8498e642-890c-419a-bb25-50fe4803fa13\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") "
Sep 05 08:15:50.231179 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231083    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-data\") pod \"8498e642-890c-419a-bb25-50fe4803fa13\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") "
Sep 05 08:15:50.231179 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231125    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmphq\" (UniqueName: \"kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-kube-api-access-dmphq\") pod \"8498e642-890c-419a-bb25-50fe4803fa13\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") "
Sep 05 08:15:50.231179 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231145    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-istio-token\") pod \"8498e642-890c-419a-bb25-50fe4803fa13\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") "
Sep 05 08:15:50.231503 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231186    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-envoy\") pod \"8498e642-890c-419a-bb25-50fe4803fa13\" (UID: \"8498e642-890c-419a-bb25-50fe4803fa13\") "
Sep 05 08:15:50.231503 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231239    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8498e642-890c-419a-bb25-50fe4803fa13" (UID: "8498e642-890c-419a-bb25-50fe4803fa13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:15:50.231503 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231296    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8498e642-890c-419a-bb25-50fe4803fa13" (UID: "8498e642-890c-419a-bb25-50fe4803fa13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:15:50.231503 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231423    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8498e642-890c-419a-bb25-50fe4803fa13-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8498e642-890c-419a-bb25-50fe4803fa13" (UID: "8498e642-890c-419a-bb25-50fe4803fa13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:15:50.231738 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231665    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-data" (OuterVolumeSpecName: "istio-data") pod "8498e642-890c-419a-bb25-50fe4803fa13" (UID: "8498e642-890c-419a-bb25-50fe4803fa13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:15:50.231738 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231727    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8498e642-890c-419a-bb25-50fe4803fa13" (UID: "8498e642-890c-419a-bb25-50fe4803fa13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:15:50.231849 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231788    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:15:50.231849 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231810    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:15:50.231849 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231825    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8498e642-890c-419a-bb25-50fe4803fa13-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:15:50.231849 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.231838    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:15:50.233616 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.233587    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-kube-api-access-dmphq" (OuterVolumeSpecName: "kube-api-access-dmphq") pod "8498e642-890c-419a-bb25-50fe4803fa13" (UID: "8498e642-890c-419a-bb25-50fe4803fa13"). InnerVolumeSpecName "kube-api-access-dmphq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:15:50.233723 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.233698    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-istio-token" (OuterVolumeSpecName: "istio-token") pod "8498e642-890c-419a-bb25-50fe4803fa13" (UID: "8498e642-890c-419a-bb25-50fe4803fa13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:15:50.234004 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.233983    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8498e642-890c-419a-bb25-50fe4803fa13-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8498e642-890c-419a-bb25-50fe4803fa13" (UID: "8498e642-890c-419a-bb25-50fe4803fa13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:15:50.234004 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.233992    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8498e642-890c-419a-bb25-50fe4803fa13" (UID: "8498e642-890c-419a-bb25-50fe4803fa13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:15:50.332645 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.332608    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmphq\" (UniqueName: \"kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-kube-api-access-dmphq\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:15:50.332645 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.332638    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8498e642-890c-419a-bb25-50fe4803fa13-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:15:50.332645 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.332649    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:15:50.332884 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.332659    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8498e642-890c-419a-bb25-50fe4803fa13-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:15:50.332884 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.332672    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8498e642-890c-419a-bb25-50fe4803fa13-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:15:50.760045 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.760011    2569 generic.go:356] "Generic (PLEG): container finished" podID="8498e642-890c-419a-bb25-50fe4803fa13" containerID="1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d" exitCode=0
Sep 05 08:15:50.760258 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.760059    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"
Sep 05 08:15:50.760258 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.760097    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" event={"ID":"8498e642-890c-419a-bb25-50fe4803fa13","Type":"ContainerDied","Data":"1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d"}
Sep 05 08:15:50.760258 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.760163    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh" event={"ID":"8498e642-890c-419a-bb25-50fe4803fa13","Type":"ContainerDied","Data":"5423a6ed68262c4dc7b8395c4aca2b1468b4530b6d7ea270d7821b9dc7477dee"}
Sep 05 08:15:50.760258 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.760194    2569 scope.go:117] "RemoveContainer" containerID="1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d"
Sep 05 08:15:50.770335 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.770318    2569 scope.go:117] "RemoveContainer" containerID="764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b"
Sep 05 08:15:50.779443 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.779210    2569 scope.go:117] "RemoveContainer" containerID="1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d"
Sep 05 08:15:50.779767 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:15:50.779736    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d\": container with ID starting with 1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d not found: ID does not exist" containerID="1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d"
Sep 05 08:15:50.779863 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.779778    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d"} err="failed to get container status \"1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d\": rpc error: code = NotFound desc = could not find container \"1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d\": container with ID starting with 1e066b50ffcc7c44cf745dd9e221eef1ee8a471159575add65ba11d7d0e5c68d not found: ID does not exist"
Sep 05 08:15:50.779863 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.779802    2569 scope.go:117] "RemoveContainer" containerID="764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b"
Sep 05 08:15:50.780112 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:15:50.780093    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b\": container with ID starting with 764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b not found: ID does not exist" containerID="764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b"
Sep 05 08:15:50.780154 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.780120    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b"} err="failed to get container status \"764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b\": rpc error: code = NotFound desc = could not find container \"764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b\": container with ID starting with 764f28130aaad26941585b5943ab2a358f9e75242d65cd817942a39f6a3d5a0b not found: ID does not exist"
Sep 05 08:15:50.782012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.781994    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"]
Sep 05 08:15:50.787958 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:50.787937    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9d46f-openshift-default-589755b94b-6ffsh"]
Sep 05 08:15:51.571298 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:15:51.571264    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8498e642-890c-419a-bb25-50fe4803fa13" path="/var/lib/kubelet/pods/8498e642-890c-419a-bb25-50fe4803fa13/volumes"
Sep 05 08:16:57.965365 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:16:57.965328    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"]
Sep 05 08:16:57.965924 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:16:57.965636    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy" containerID="cri-o://6f58da5f5d605afd2cc07ad9c7d9d25b5e4f59407a0046b6f22ebaf1eaa1996f" gracePeriod=30
Sep 05 08:17:00.416919 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:00.416833    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 05 08:17:00.416919 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:00.416897    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 05 08:17:03.186733 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.186698    2569 generic.go:356] "Generic (PLEG): container finished" podID="e7553196-e42d-403c-a14d-d9030613aa1f" containerID="6f58da5f5d605afd2cc07ad9c7d9d25b5e4f59407a0046b6f22ebaf1eaa1996f" exitCode=0
Sep 05 08:17:03.187201 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.186773    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" event={"ID":"e7553196-e42d-403c-a14d-d9030613aa1f","Type":"ContainerDied","Data":"6f58da5f5d605afd2cc07ad9c7d9d25b5e4f59407a0046b6f22ebaf1eaa1996f"}
Sep 05 08:17:03.187201 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.186835    2569 scope.go:117] "RemoveContainer" containerID="3b9d77dd8f53d65f662e79f0945663e0d7d2ed65fa4bdb75ed57a96628733d65"
Sep 05 08:17:03.225409 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.225384    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:17:03.281867 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.281781    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-data\") pod \"e7553196-e42d-403c-a14d-d9030613aa1f\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") "
Sep 05 08:17:03.281867 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.281818    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7553196-e42d-403c-a14d-d9030613aa1f-istiod-ca-cert\") pod \"e7553196-e42d-403c-a14d-d9030613aa1f\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") "
Sep 05 08:17:03.281867 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.281846    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-istio-token\") pod \"e7553196-e42d-403c-a14d-d9030613aa1f\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") "
Sep 05 08:17:03.282163 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.281873    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-socket\") pod \"e7553196-e42d-403c-a14d-d9030613aa1f\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") "
Sep 05 08:17:03.282163 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.281912    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-certs\") pod \"e7553196-e42d-403c-a14d-d9030613aa1f\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") "
Sep 05 08:17:03.282163 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.281965    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7553196-e42d-403c-a14d-d9030613aa1f-istio-podinfo\") pod \"e7553196-e42d-403c-a14d-d9030613aa1f\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") "
Sep 05 08:17:03.282163 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282018    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9c9c\" (UniqueName: \"kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-kube-api-access-j9c9c\") pod \"e7553196-e42d-403c-a14d-d9030613aa1f\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") "
Sep 05 08:17:03.282163 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282055    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-credential-socket\") pod \"e7553196-e42d-403c-a14d-d9030613aa1f\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") "
Sep 05 08:17:03.282163 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282068    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e7553196-e42d-403c-a14d-d9030613aa1f" (UID: "e7553196-e42d-403c-a14d-d9030613aa1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:03.282163 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282149    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-envoy\") pod \"e7553196-e42d-403c-a14d-d9030613aa1f\" (UID: \"e7553196-e42d-403c-a14d-d9030613aa1f\") "
Sep 05 08:17:03.282558 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282274    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e7553196-e42d-403c-a14d-d9030613aa1f" (UID: "e7553196-e42d-403c-a14d-d9030613aa1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:03.282558 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282298    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7553196-e42d-403c-a14d-d9030613aa1f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e7553196-e42d-403c-a14d-d9030613aa1f" (UID: "e7553196-e42d-403c-a14d-d9030613aa1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:17:03.282558 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282477    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e7553196-e42d-403c-a14d-d9030613aa1f" (UID: "e7553196-e42d-403c-a14d-d9030613aa1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:03.282899 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282721    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:03.282899 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282740    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7553196-e42d-403c-a14d-d9030613aa1f-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:03.282899 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282753    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:03.282899 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282767    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:03.282899 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.282785    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e7553196-e42d-403c-a14d-d9030613aa1f" (UID: "e7553196-e42d-403c-a14d-d9030613aa1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:03.284288 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.284265    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7553196-e42d-403c-a14d-d9030613aa1f" (UID: "e7553196-e42d-403c-a14d-d9030613aa1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:03.284441 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.284412    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-kube-api-access-j9c9c" (OuterVolumeSpecName: "kube-api-access-j9c9c") pod "e7553196-e42d-403c-a14d-d9030613aa1f" (UID: "e7553196-e42d-403c-a14d-d9030613aa1f"). InnerVolumeSpecName "kube-api-access-j9c9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:03.284513 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.284481    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e7553196-e42d-403c-a14d-d9030613aa1f" (UID: "e7553196-e42d-403c-a14d-d9030613aa1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:03.284916 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.284901    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7553196-e42d-403c-a14d-d9030613aa1f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e7553196-e42d-403c-a14d-d9030613aa1f" (UID: "e7553196-e42d-403c-a14d-d9030613aa1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:17:03.383990 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.383959    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9c9c\" (UniqueName: \"kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-kube-api-access-j9c9c\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:03.383990 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.383987    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:03.383990 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.383999    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7553196-e42d-403c-a14d-d9030613aa1f-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:03.384225 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.384008    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7553196-e42d-403c-a14d-d9030613aa1f-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:03.384225 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:03.384016    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7553196-e42d-403c-a14d-d9030613aa1f-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:04.193226 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:04.193192    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt" event={"ID":"e7553196-e42d-403c-a14d-d9030613aa1f","Type":"ContainerDied","Data":"0601f2cc4737d108056c67d67a72f9ed991eceb2dd091acd83b7a89f7fb132d9"}
Sep 05 08:17:04.193226 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:04.193218    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"
Sep 05 08:17:04.193731 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:04.193249    2569 scope.go:117] "RemoveContainer" containerID="6f58da5f5d605afd2cc07ad9c7d9d25b5e4f59407a0046b6f22ebaf1eaa1996f"
Sep 05 08:17:04.212819 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:04.212794    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"]
Sep 05 08:17:04.216652 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:04.216628    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-3d4dc1-openshift-default-694fd9d968-clndt"]
Sep 05 08:17:05.570675 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:05.570643    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" path="/var/lib/kubelet/pods/e7553196-e42d-403c-a14d-d9030613aa1f/volumes"
Sep 05 08:17:08.802407 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:08.802376    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"]
Sep 05 08:17:08.802904 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:08.802664    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5" podUID="8f61bbde-f924-475c-9854-98de189e7ef3" containerName="maas-api" containerID="cri-o://338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4" gracePeriod=30
Sep 05 08:17:09.068212 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.068194    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:17:09.141263 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.141224    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f61bbde-f924-475c-9854-98de189e7ef3-openshift-service-ca\") pod \"8f61bbde-f924-475c-9854-98de189e7ef3\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") "
Sep 05 08:17:09.141454 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.141278    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-maas-api-tls\") pod \"8f61bbde-f924-475c-9854-98de189e7ef3\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") "
Sep 05 08:17:09.141454 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.141341    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkc5l\" (UniqueName: \"kubernetes.io/projected/8f61bbde-f924-475c-9854-98de189e7ef3-kube-api-access-tkc5l\") pod \"8f61bbde-f924-475c-9854-98de189e7ef3\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") "
Sep 05 08:17:09.141454 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.141368    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-metrics-certs\") pod \"8f61bbde-f924-475c-9854-98de189e7ef3\" (UID: \"8f61bbde-f924-475c-9854-98de189e7ef3\") "
Sep 05 08:17:09.141662 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.141641    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f61bbde-f924-475c-9854-98de189e7ef3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8f61bbde-f924-475c-9854-98de189e7ef3" (UID: "8f61bbde-f924-475c-9854-98de189e7ef3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:17:09.141860 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.141837    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f61bbde-f924-475c-9854-98de189e7ef3-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:09.143843 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.143807    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f61bbde-f924-475c-9854-98de189e7ef3-kube-api-access-tkc5l" (OuterVolumeSpecName: "kube-api-access-tkc5l") pod "8f61bbde-f924-475c-9854-98de189e7ef3" (UID: "8f61bbde-f924-475c-9854-98de189e7ef3"). InnerVolumeSpecName "kube-api-access-tkc5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:09.144125 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.144095    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8f61bbde-f924-475c-9854-98de189e7ef3" (UID: "8f61bbde-f924-475c-9854-98de189e7ef3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:17:09.144240 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.144119    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8f61bbde-f924-475c-9854-98de189e7ef3" (UID: "8f61bbde-f924-475c-9854-98de189e7ef3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:17:09.222589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.222513    2569 generic.go:356] "Generic (PLEG): container finished" podID="8f61bbde-f924-475c-9854-98de189e7ef3" containerID="338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4" exitCode=0
Sep 05 08:17:09.222589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.222596    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"
Sep 05 08:17:09.222852 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.222601    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5" event={"ID":"8f61bbde-f924-475c-9854-98de189e7ef3","Type":"ContainerDied","Data":"338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4"}
Sep 05 08:17:09.222852 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.222672    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5" event={"ID":"8f61bbde-f924-475c-9854-98de189e7ef3","Type":"ContainerDied","Data":"bc837e26cec2f0d1974f9fefcaf9cc9e635dbcbe96de4852922649ea4f8f9579"}
Sep 05 08:17:09.222852 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.222703    2569 scope.go:117] "RemoveContainer" containerID="338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4"
Sep 05 08:17:09.232748 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.232732    2569 scope.go:117] "RemoveContainer" containerID="338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4"
Sep 05 08:17:09.233033 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:17:09.233011    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4\": container with ID starting with 338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4 not found: ID does not exist" containerID="338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4"
Sep 05 08:17:09.233130 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.233039    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4"} err="failed to get container status \"338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4\": rpc error: code = NotFound desc = could not find container \"338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4\": container with ID starting with 338ae17ab950329deef6530ed377b64e98a04304e998f310995e6c82692eeeb4 not found: ID does not exist"
Sep 05 08:17:09.243152 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.243126    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:09.243152 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.243152    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkc5l\" (UniqueName: \"kubernetes.io/projected/8f61bbde-f924-475c-9854-98de189e7ef3-kube-api-access-tkc5l\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:09.243337 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.243161    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f61bbde-f924-475c-9854-98de189e7ef3-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:09.244076 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.244052    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"]
Sep 05 08:17:09.248147 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.248125    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d14aea-6db5c594bf-msrj5"]
Sep 05 08:17:09.577951 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:09.577908    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f61bbde-f924-475c-9854-98de189e7ef3" path="/var/lib/kubelet/pods/8f61bbde-f924-475c-9854-98de189e7ef3/volumes"
Sep 05 08:17:15.403596 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:15.403560    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"]
Sep 05 08:17:15.404096 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:15.403814    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy" containerID="cri-o://a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af" gracePeriod=30
Sep 05 08:17:20.668477 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.668417    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:17:20.761666 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.761620    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25866f6f-8128-42d2-b896-24e5e778a6d0-istiod-ca-cert\") pod \"25866f6f-8128-42d2-b896-24e5e778a6d0\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") "
Sep 05 08:17:20.761666 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.761671    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-podinfo\") pod \"25866f6f-8128-42d2-b896-24e5e778a6d0\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") "
Sep 05 08:17:20.761887 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.761714    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-envoy\") pod \"25866f6f-8128-42d2-b896-24e5e778a6d0\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") "
Sep 05 08:17:20.761887 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.761750    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnnl7\" (UniqueName: \"kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-kube-api-access-wnnl7\") pod \"25866f6f-8128-42d2-b896-24e5e778a6d0\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") "
Sep 05 08:17:20.761887 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.761764    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-token\") pod \"25866f6f-8128-42d2-b896-24e5e778a6d0\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") "
Sep 05 08:17:20.762041 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.761916    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-data\") pod \"25866f6f-8128-42d2-b896-24e5e778a6d0\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") "
Sep 05 08:17:20.762041 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.761987    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-credential-socket\") pod \"25866f6f-8128-42d2-b896-24e5e778a6d0\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") "
Sep 05 08:17:20.762150 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762056    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-certs\") pod \"25866f6f-8128-42d2-b896-24e5e778a6d0\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") "
Sep 05 08:17:20.762150 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762066    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25866f6f-8128-42d2-b896-24e5e778a6d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "25866f6f-8128-42d2-b896-24e5e778a6d0" (UID: "25866f6f-8128-42d2-b896-24e5e778a6d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:17:20.762150 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762135    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-socket\") pod \"25866f6f-8128-42d2-b896-24e5e778a6d0\" (UID: \"25866f6f-8128-42d2-b896-24e5e778a6d0\") "
Sep 05 08:17:20.762315 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762193    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "25866f6f-8128-42d2-b896-24e5e778a6d0" (UID: "25866f6f-8128-42d2-b896-24e5e778a6d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:20.762315 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762263    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "25866f6f-8128-42d2-b896-24e5e778a6d0" (UID: "25866f6f-8128-42d2-b896-24e5e778a6d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:20.762434 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762403    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "25866f6f-8128-42d2-b896-24e5e778a6d0" (UID: "25866f6f-8128-42d2-b896-24e5e778a6d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:20.762689 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762654    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "25866f6f-8128-42d2-b896-24e5e778a6d0" (UID: "25866f6f-8128-42d2-b896-24e5e778a6d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:20.762817 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762799    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:20.762878 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762824    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:20.762878 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762839    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:20.762878 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762852    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:20.762878 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.762865    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25866f6f-8128-42d2-b896-24e5e778a6d0-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:20.764194 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.764171    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "25866f6f-8128-42d2-b896-24e5e778a6d0" (UID: "25866f6f-8128-42d2-b896-24e5e778a6d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:20.764324 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.764193    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "25866f6f-8128-42d2-b896-24e5e778a6d0" (UID: "25866f6f-8128-42d2-b896-24e5e778a6d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:20.764324 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.764225    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "25866f6f-8128-42d2-b896-24e5e778a6d0" (UID: "25866f6f-8128-42d2-b896-24e5e778a6d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:17:20.764589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.764572    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-kube-api-access-wnnl7" (OuterVolumeSpecName: "kube-api-access-wnnl7") pod "25866f6f-8128-42d2-b896-24e5e778a6d0" (UID: "25866f6f-8128-42d2-b896-24e5e778a6d0"). InnerVolumeSpecName "kube-api-access-wnnl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:20.863971 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.863915    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:20.863971 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.863963    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:20.863971 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.863976    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnnl7\" (UniqueName: \"kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-kube-api-access-wnnl7\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:20.864212 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:20.863987    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25866f6f-8128-42d2-b896-24e5e778a6d0-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:21.294388 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.294354    2569 generic.go:356] "Generic (PLEG): container finished" podID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerID="a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af" exitCode=0
Sep 05 08:17:21.294577 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.294401    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"
Sep 05 08:17:21.294577 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.294435    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" event={"ID":"25866f6f-8128-42d2-b896-24e5e778a6d0","Type":"ContainerDied","Data":"a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af"}
Sep 05 08:17:21.294577 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.294488    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp" event={"ID":"25866f6f-8128-42d2-b896-24e5e778a6d0","Type":"ContainerDied","Data":"cf6437c16928e262914dfaf41bdfe8bec99d083a7aaa09caec67b682bada5ee5"}
Sep 05 08:17:21.294577 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.294510    2569 scope.go:117] "RemoveContainer" containerID="a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af"
Sep 05 08:17:21.304790 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.304772    2569 scope.go:117] "RemoveContainer" containerID="0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353"
Sep 05 08:17:21.313319 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.313302    2569 scope.go:117] "RemoveContainer" containerID="a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af"
Sep 05 08:17:21.313643 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:17:21.313615    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af\": container with ID starting with a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af not found: ID does not exist" containerID="a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af"
Sep 05 08:17:21.313704 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.313652    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af"} err="failed to get container status \"a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af\": rpc error: code = NotFound desc = could not find container \"a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af\": container with ID starting with a13443969c20e084a1f2cbd4156ea8a6a4c90fd3b4803763e7619ba3258ce6af not found: ID does not exist"
Sep 05 08:17:21.313704 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.313668    2569 scope.go:117] "RemoveContainer" containerID="0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353"
Sep 05 08:17:21.313929 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:17:21.313910    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353\": container with ID starting with 0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353 not found: ID does not exist" containerID="0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353"
Sep 05 08:17:21.314001 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.313938    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353"} err="failed to get container status \"0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353\": rpc error: code = NotFound desc = could not find container \"0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353\": container with ID starting with 0a9af7cb3d2f14bccc4f2dab194397a4004b145ead0136e68836ca1ae1879353 not found: ID does not exist"
Sep 05 08:17:21.317694 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.317669    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"]
Sep 05 08:17:21.321649 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.321630    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-d14aea-openshift-default-6b4c475cd4-9gzsp"]
Sep 05 08:17:21.571654 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:21.571619    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" path="/var/lib/kubelet/pods/25866f6f-8128-42d2-b896-24e5e778a6d0/volumes"
Sep 05 08:17:32.958644 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:32.958607    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"]
Sep 05 08:17:32.959201 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:32.958921    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy" containerID="cri-o://afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78" gracePeriod=30
Sep 05 08:17:38.216847 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.216815    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:17:38.333492 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.333462    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-podinfo\") pod \"d06e4fd8-2449-4803-a7ab-fc71b8430625\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") "
Sep 05 08:17:38.333704 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.333501    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-token\") pod \"d06e4fd8-2449-4803-a7ab-fc71b8430625\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") "
Sep 05 08:17:38.333704 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.333548    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-credential-socket\") pod \"d06e4fd8-2449-4803-a7ab-fc71b8430625\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") "
Sep 05 08:17:38.333704 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.333569    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d06e4fd8-2449-4803-a7ab-fc71b8430625-istiod-ca-cert\") pod \"d06e4fd8-2449-4803-a7ab-fc71b8430625\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") "
Sep 05 08:17:38.333704 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.333606    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-certs\") pod \"d06e4fd8-2449-4803-a7ab-fc71b8430625\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") "
Sep 05 08:17:38.333704 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.333636    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv5xs\" (UniqueName: \"kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-kube-api-access-pv5xs\") pod \"d06e4fd8-2449-4803-a7ab-fc71b8430625\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") "
Sep 05 08:17:38.333704 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.333698    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-data\") pod \"d06e4fd8-2449-4803-a7ab-fc71b8430625\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") "
Sep 05 08:17:38.334012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.333745    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-envoy\") pod \"d06e4fd8-2449-4803-a7ab-fc71b8430625\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") "
Sep 05 08:17:38.334012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.333768    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-socket\") pod \"d06e4fd8-2449-4803-a7ab-fc71b8430625\" (UID: \"d06e4fd8-2449-4803-a7ab-fc71b8430625\") "
Sep 05 08:17:38.334012 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.333932    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d06e4fd8-2449-4803-a7ab-fc71b8430625" (UID: "d06e4fd8-2449-4803-a7ab-fc71b8430625"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:38.334174 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.334069    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-data" (OuterVolumeSpecName: "istio-data") pod "d06e4fd8-2449-4803-a7ab-fc71b8430625" (UID: "d06e4fd8-2449-4803-a7ab-fc71b8430625"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:38.334174 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.334146    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d06e4fd8-2449-4803-a7ab-fc71b8430625" (UID: "d06e4fd8-2449-4803-a7ab-fc71b8430625"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:38.334283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.334182    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-data\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:38.334283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.334203    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-credential-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:38.334283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.334181    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d06e4fd8-2449-4803-a7ab-fc71b8430625-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d06e4fd8-2449-4803-a7ab-fc71b8430625" (UID: "d06e4fd8-2449-4803-a7ab-fc71b8430625"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:17:38.334283 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.334243    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d06e4fd8-2449-4803-a7ab-fc71b8430625" (UID: "d06e4fd8-2449-4803-a7ab-fc71b8430625"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:38.336098 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.336060    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d06e4fd8-2449-4803-a7ab-fc71b8430625" (UID: "d06e4fd8-2449-4803-a7ab-fc71b8430625"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 08:17:38.336299 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.336282    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-kube-api-access-pv5xs" (OuterVolumeSpecName: "kube-api-access-pv5xs") pod "d06e4fd8-2449-4803-a7ab-fc71b8430625" (UID: "d06e4fd8-2449-4803-a7ab-fc71b8430625"). InnerVolumeSpecName "kube-api-access-pv5xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:38.336373 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.336346    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d06e4fd8-2449-4803-a7ab-fc71b8430625" (UID: "d06e4fd8-2449-4803-a7ab-fc71b8430625"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 08:17:38.336419 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.336380    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-token" (OuterVolumeSpecName: "istio-token") pod "d06e4fd8-2449-4803-a7ab-fc71b8430625" (UID: "d06e4fd8-2449-4803-a7ab-fc71b8430625"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:38.398885 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.398849    2569 generic.go:356] "Generic (PLEG): container finished" podID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerID="afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78" exitCode=0
Sep 05 08:17:38.399055 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.398900    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"
Sep 05 08:17:38.399055 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.398929    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" event={"ID":"d06e4fd8-2449-4803-a7ab-fc71b8430625","Type":"ContainerDied","Data":"afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78"}
Sep 05 08:17:38.399055 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.398981    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj" event={"ID":"d06e4fd8-2449-4803-a7ab-fc71b8430625","Type":"ContainerDied","Data":"d56b1f3a93a71abcd65699a3779b9f1b19113b864b77af49b9aae5f84db0030f"}
Sep 05 08:17:38.399055 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.399003    2569 scope.go:117] "RemoveContainer" containerID="afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78"
Sep 05 08:17:38.409282 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.409264    2569 scope.go:117] "RemoveContainer" containerID="8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db"
Sep 05 08:17:38.417627 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.417610    2569 scope.go:117] "RemoveContainer" containerID="afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78"
Sep 05 08:17:38.417886 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:17:38.417868    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78\": container with ID starting with afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78 not found: ID does not exist" containerID="afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78"
Sep 05 08:17:38.417929 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.417896    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78"} err="failed to get container status \"afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78\": rpc error: code = NotFound desc = could not find container \"afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78\": container with ID starting with afe4ff3400525bbf8131c3d78a90fb474977220cf99d98a227b49737a6e56e78 not found: ID does not exist"
Sep 05 08:17:38.417929 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.417915    2569 scope.go:117] "RemoveContainer" containerID="8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db"
Sep 05 08:17:38.418266 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:17:38.418242    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db\": container with ID starting with 8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db not found: ID does not exist" containerID="8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db"
Sep 05 08:17:38.418383 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.418271    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db"} err="failed to get container status \"8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db\": rpc error: code = NotFound desc = could not find container \"8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db\": container with ID starting with 8c08f5e7ab6a61351814ce022ee981d27654df2b3b91e1aefd0921d329f363db not found: ID does not exist"
Sep 05 08:17:38.422179 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.422160    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"]
Sep 05 08:17:38.426141 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.426120    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8c2e20-openshift-default-69495bf976-jxlsj"]
Sep 05 08:17:38.435339 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.435319    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-envoy\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:38.435435 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.435342    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-socket\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:38.435435 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.435355    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-podinfo\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:38.435435 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.435368    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-istio-token\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:38.435435 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.435379    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d06e4fd8-2449-4803-a7ab-fc71b8430625-istiod-ca-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:38.435435 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.435391    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d06e4fd8-2449-4803-a7ab-fc71b8430625-workload-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:38.435435 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:38.435404    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv5xs\" (UniqueName: \"kubernetes.io/projected/d06e4fd8-2449-4803-a7ab-fc71b8430625-kube-api-access-pv5xs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:39.572017 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:39.571983    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" path="/var/lib/kubelet/pods/d06e4fd8-2449-4803-a7ab-fc71b8430625/volumes"
Sep 05 08:17:41.611062 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:41.611026    2569 scope.go:117] "RemoveContainer" containerID="2b494eace076645d0dccca9fd739f9ca5dee93c85bcf772941e81ac422c94847"
Sep 05 08:17:43.721306 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:43.721272    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"]
Sep 05 08:17:43.722125 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:43.722087    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv" podUID="7d87658e-9333-4948-aa74-637996ab1c4b" containerName="maas-api" containerID="cri-o://dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24" gracePeriod=30
Sep 05 08:17:43.980859 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:43.980795    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:17:44.089425 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.089393    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d87658e-9333-4948-aa74-637996ab1c4b-openshift-service-ca\") pod \"7d87658e-9333-4948-aa74-637996ab1c4b\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") "
Sep 05 08:17:44.089639 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.089511    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65h6d\" (UniqueName: \"kubernetes.io/projected/7d87658e-9333-4948-aa74-637996ab1c4b-kube-api-access-65h6d\") pod \"7d87658e-9333-4948-aa74-637996ab1c4b\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") "
Sep 05 08:17:44.089639 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.089558    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-maas-api-tls\") pod \"7d87658e-9333-4948-aa74-637996ab1c4b\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") "
Sep 05 08:17:44.089639 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.089590    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-metrics-certs\") pod \"7d87658e-9333-4948-aa74-637996ab1c4b\" (UID: \"7d87658e-9333-4948-aa74-637996ab1c4b\") "
Sep 05 08:17:44.089828 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.089805    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d87658e-9333-4948-aa74-637996ab1c4b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d87658e-9333-4948-aa74-637996ab1c4b" (UID: "7d87658e-9333-4948-aa74-637996ab1c4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:17:44.090010 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.089974    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d87658e-9333-4948-aa74-637996ab1c4b-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:44.092147 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.092120    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d87658e-9333-4948-aa74-637996ab1c4b-kube-api-access-65h6d" (OuterVolumeSpecName: "kube-api-access-65h6d") pod "7d87658e-9333-4948-aa74-637996ab1c4b" (UID: "7d87658e-9333-4948-aa74-637996ab1c4b"). InnerVolumeSpecName "kube-api-access-65h6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:44.092444 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.092404    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7d87658e-9333-4948-aa74-637996ab1c4b" (UID: "7d87658e-9333-4948-aa74-637996ab1c4b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:17:44.092444 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.092427    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d87658e-9333-4948-aa74-637996ab1c4b" (UID: "7d87658e-9333-4948-aa74-637996ab1c4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:17:44.191038 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.190994    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65h6d\" (UniqueName: \"kubernetes.io/projected/7d87658e-9333-4948-aa74-637996ab1c4b-kube-api-access-65h6d\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:44.191038 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.191032    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:44.191038 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.191042    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d87658e-9333-4948-aa74-637996ab1c4b-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:44.442062 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.442026    2569 generic.go:356] "Generic (PLEG): container finished" podID="7d87658e-9333-4948-aa74-637996ab1c4b" containerID="dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24" exitCode=0
Sep 05 08:17:44.442245 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.442067    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"
Sep 05 08:17:44.442245 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.442109    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv" event={"ID":"7d87658e-9333-4948-aa74-637996ab1c4b","Type":"ContainerDied","Data":"dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24"}
Sep 05 08:17:44.442245 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.442169    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv" event={"ID":"7d87658e-9333-4948-aa74-637996ab1c4b","Type":"ContainerDied","Data":"33ca5cafab9ad410166a31ff48379e7f75ac2e0dbf0c34b74cf6f20aa3ea4237"}
Sep 05 08:17:44.442245 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.442200    2569 scope.go:117] "RemoveContainer" containerID="dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24"
Sep 05 08:17:44.452280 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.452261    2569 scope.go:117] "RemoveContainer" containerID="dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24"
Sep 05 08:17:44.452573 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:17:44.452552    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24\": container with ID starting with dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24 not found: ID does not exist" containerID="dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24"
Sep 05 08:17:44.452645 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.452582    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24"} err="failed to get container status \"dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24\": rpc error: code = NotFound desc = could not find container \"dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24\": container with ID starting with dba0e46364d3635b4cb763f54406e04a0691125df65125821f91fc7d8dcc6b24 not found: ID does not exist"
Sep 05 08:17:44.464444 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.464413    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"]
Sep 05 08:17:44.468099 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:44.468080    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5ef325-647b58dd8-7zrcv"]
Sep 05 08:17:45.575731 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:45.573616    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d87658e-9333-4948-aa74-637996ab1c4b" path="/var/lib/kubelet/pods/7d87658e-9333-4948-aa74-637996ab1c4b/volumes"
Sep 05 08:17:53.035911 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.035874    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-jbvd9"]
Sep 05 08:17:53.036318 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.036148    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9" podUID="d730375f-48bd-48ef-b0b3-533cf1fdfba7" containerName="manager" containerID="cri-o://ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e" gracePeriod=30
Sep 05 08:17:53.041271 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.041246    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9" podUID="d730375f-48bd-48ef-b0b3-533cf1fdfba7" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.41:8081/readyz\": dial tcp 10.134.0.41:8081: connect: connection refused"
Sep 05 08:17:53.297381 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.297321    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 08:17:53.385249 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.385216    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jfp5\" (UniqueName: \"kubernetes.io/projected/d730375f-48bd-48ef-b0b3-533cf1fdfba7-kube-api-access-5jfp5\") pod \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") "
Sep 05 08:17:53.385423 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.385295    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-metrics-certs\") pod \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") "
Sep 05 08:17:53.385423 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.385390    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-webhook-cert\") pod \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\" (UID: \"d730375f-48bd-48ef-b0b3-533cf1fdfba7\") "
Sep 05 08:17:53.387498 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.387471    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d730375f-48bd-48ef-b0b3-533cf1fdfba7" (UID: "d730375f-48bd-48ef-b0b3-533cf1fdfba7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:17:53.387947 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.387924    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d730375f-48bd-48ef-b0b3-533cf1fdfba7" (UID: "d730375f-48bd-48ef-b0b3-533cf1fdfba7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:17:53.387947 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.387940    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d730375f-48bd-48ef-b0b3-533cf1fdfba7-kube-api-access-5jfp5" (OuterVolumeSpecName: "kube-api-access-5jfp5") pod "d730375f-48bd-48ef-b0b3-533cf1fdfba7" (UID: "d730375f-48bd-48ef-b0b3-533cf1fdfba7"). InnerVolumeSpecName "kube-api-access-5jfp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:17:53.486198 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.486165    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jfp5\" (UniqueName: \"kubernetes.io/projected/d730375f-48bd-48ef-b0b3-533cf1fdfba7-kube-api-access-5jfp5\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:53.486198 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.486192    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:53.486198 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.486202    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d730375f-48bd-48ef-b0b3-533cf1fdfba7-webhook-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:17:53.490279 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.490248    2569 generic.go:356] "Generic (PLEG): container finished" podID="d730375f-48bd-48ef-b0b3-533cf1fdfba7" containerID="ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e" exitCode=0
Sep 05 08:17:53.490412 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.490293    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9"
Sep 05 08:17:53.490412 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.490339    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9" event={"ID":"d730375f-48bd-48ef-b0b3-533cf1fdfba7","Type":"ContainerDied","Data":"ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e"}
Sep 05 08:17:53.490412 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.490395    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb86cc4bd-jbvd9" event={"ID":"d730375f-48bd-48ef-b0b3-533cf1fdfba7","Type":"ContainerDied","Data":"f13075ebd359607e2a73354b623b127c9af6105d9261bc31894d31475b0cc3a0"}
Sep 05 08:17:53.490512 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.490426    2569 scope.go:117] "RemoveContainer" containerID="ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e"
Sep 05 08:17:53.500752 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.500735    2569 scope.go:117] "RemoveContainer" containerID="ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e"
Sep 05 08:17:53.501025 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:17:53.501004    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e\": container with ID starting with ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e not found: ID does not exist" containerID="ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e"
Sep 05 08:17:53.501092 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.501036    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e"} err="failed to get container status \"ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e\": rpc error: code = NotFound desc = could not find container \"ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e\": container with ID starting with ac018b486b7cc19c81e32ef5e033144d81657001aa6b9878e92259930b42df6e not found: ID does not exist"
Sep 05 08:17:53.511518 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.511482    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-jbvd9"]
Sep 05 08:17:53.515557 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.515538    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-jbvd9"]
Sep 05 08:17:53.570624 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:17:53.570591    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d730375f-48bd-48ef-b0b3-533cf1fdfba7" path="/var/lib/kubelet/pods/d730375f-48bd-48ef-b0b3-533cf1fdfba7/volumes"
Sep 05 08:18:39.704837 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:39.704803    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:18:39.711275 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:39.711251    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:18:54.309765 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.309716    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-2xm9q"]
Sep 05 08:18:54.310244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310176    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d730375f-48bd-48ef-b0b3-533cf1fdfba7" containerName="manager"
Sep 05 08:18:54.310244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310190    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d730375f-48bd-48ef-b0b3-533cf1fdfba7" containerName="manager"
Sep 05 08:18:54.310244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310204    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy"
Sep 05 08:18:54.310244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310209    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy"
Sep 05 08:18:54.310244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310217    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f61bbde-f924-475c-9854-98de189e7ef3" containerName="maas-api"
Sep 05 08:18:54.310244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310223    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f61bbde-f924-475c-9854-98de189e7ef3" containerName="maas-api"
Sep 05 08:18:54.310244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310233    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d87658e-9333-4948-aa74-637996ab1c4b" containerName="maas-api"
Sep 05 08:18:54.310244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310238    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d87658e-9333-4948-aa74-637996ab1c4b" containerName="maas-api"
Sep 05 08:18:54.310244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310244    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy"
Sep 05 08:18:54.310244 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310249    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310261    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310266    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310272    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310277    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310285    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310290    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310303    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310308    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310314    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310319    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310325    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310330    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310388    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f61bbde-f924-475c-9854-98de189e7ef3" containerName="maas-api"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310395    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310400    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310404    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310410    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d87658e-9333-4948-aa74-637996ab1c4b" containerName="maas-api"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310418    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d730375f-48bd-48ef-b0b3-533cf1fdfba7" containerName="manager"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310424    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7553196-e42d-403c-a14d-d9030613aa1f" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310431    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310435    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8498e642-890c-419a-bb25-50fe4803fa13" containerName="istio-proxy"
Sep 05 08:18:54.310589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.310441    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06e4fd8-2449-4803-a7ab-fc71b8430625" containerName="istio-proxy"
Sep 05 08:18:54.313629 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.313613    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.317938 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.317914    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 05 08:18:54.317938 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.317929    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9qpdw\""
Sep 05 08:18:54.318171 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.317930    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 05 08:18:54.318171 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.318106    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 05 08:18:54.323769 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.323745    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-2xm9q"]
Sep 05 08:18:54.380807 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.380772    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7lkx\" (UniqueName: \"kubernetes.io/projected/6d4884a1-4464-4f94-a336-d62ef88a777b-kube-api-access-f7lkx\") pod \"maas-controller-7cb86cc4bd-2xm9q\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") " pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.380991 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.380841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-webhook-cert\") pod \"maas-controller-7cb86cc4bd-2xm9q\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") " pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.380991 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.380887    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-metrics-certs\") pod \"maas-controller-7cb86cc4bd-2xm9q\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") " pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.481594 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.481555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7lkx\" (UniqueName: \"kubernetes.io/projected/6d4884a1-4464-4f94-a336-d62ef88a777b-kube-api-access-f7lkx\") pod \"maas-controller-7cb86cc4bd-2xm9q\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") " pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.481785 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.481639    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-webhook-cert\") pod \"maas-controller-7cb86cc4bd-2xm9q\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") " pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.481785 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.481679    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-metrics-certs\") pod \"maas-controller-7cb86cc4bd-2xm9q\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") " pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.484254 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.484214    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-metrics-certs\") pod \"maas-controller-7cb86cc4bd-2xm9q\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") " pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.484375 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.484289    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-webhook-cert\") pod \"maas-controller-7cb86cc4bd-2xm9q\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") " pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.489846 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.489823    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7lkx\" (UniqueName: \"kubernetes.io/projected/6d4884a1-4464-4f94-a336-d62ef88a777b-kube-api-access-f7lkx\") pod \"maas-controller-7cb86cc4bd-2xm9q\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") " pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.625063 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.625030    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:54.895664 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.895639    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-2xm9q"]
Sep 05 08:18:54.897599 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:18:54.897577    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4884a1_4464_4f94_a336_d62ef88a777b.slice/crio-e7f78949d1c56da7724077c6c8dae85dde640861fa280f0285ce6e1a27b0e8fc WatchSource:0}: Error finding container e7f78949d1c56da7724077c6c8dae85dde640861fa280f0285ce6e1a27b0e8fc: Status 404 returned error can't find the container with id e7f78949d1c56da7724077c6c8dae85dde640861fa280f0285ce6e1a27b0e8fc
Sep 05 08:18:54.899100 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:54.899083    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 08:18:55.825188 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:55.825147    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q" event={"ID":"6d4884a1-4464-4f94-a336-d62ef88a777b","Type":"ContainerStarted","Data":"1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109"}
Sep 05 08:18:55.825671 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:55.825224    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:18:55.825671 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:55.825247    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q" event={"ID":"6d4884a1-4464-4f94-a336-d62ef88a777b","Type":"ContainerStarted","Data":"e7f78949d1c56da7724077c6c8dae85dde640861fa280f0285ce6e1a27b0e8fc"}
Sep 05 08:18:55.844065 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:18:55.844009    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q" podStartSLOduration=1.187619947 podStartE2EDuration="1.843993248s" totalImagesPullingTime="656.373301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:18:55.842323245 +0000 UTC m=+1816.787645050" watchObservedRunningTime="2026-09-05 08:18:55.843993248 +0000 UTC m=+1816.789315052"
Sep 05 08:19:06.832923 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:06.832891    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:19:14.084676 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.084643    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-2xm9q"]
Sep 05 08:19:14.085086 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.084858    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q" podUID="6d4884a1-4464-4f94-a336-d62ef88a777b" containerName="manager" containerID="cri-o://1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109" gracePeriod=30
Sep 05 08:19:14.327499 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.327472    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:19:14.485019 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.484923    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-webhook-cert\") pod \"6d4884a1-4464-4f94-a336-d62ef88a777b\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") "
Sep 05 08:19:14.485019 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.484992    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-metrics-certs\") pod \"6d4884a1-4464-4f94-a336-d62ef88a777b\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") "
Sep 05 08:19:14.485261 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.485039    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7lkx\" (UniqueName: \"kubernetes.io/projected/6d4884a1-4464-4f94-a336-d62ef88a777b-kube-api-access-f7lkx\") pod \"6d4884a1-4464-4f94-a336-d62ef88a777b\" (UID: \"6d4884a1-4464-4f94-a336-d62ef88a777b\") "
Sep 05 08:19:14.487233 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.487197    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6d4884a1-4464-4f94-a336-d62ef88a777b" (UID: "6d4884a1-4464-4f94-a336-d62ef88a777b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:19:14.487233 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.487219    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d4884a1-4464-4f94-a336-d62ef88a777b-kube-api-access-f7lkx" (OuterVolumeSpecName: "kube-api-access-f7lkx") pod "6d4884a1-4464-4f94-a336-d62ef88a777b" (UID: "6d4884a1-4464-4f94-a336-d62ef88a777b"). InnerVolumeSpecName "kube-api-access-f7lkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:19:14.487385 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.487227    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6d4884a1-4464-4f94-a336-d62ef88a777b" (UID: "6d4884a1-4464-4f94-a336-d62ef88a777b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:19:14.586342 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.586311    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-webhook-cert\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:19:14.586342 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.586338    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d4884a1-4464-4f94-a336-d62ef88a777b-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:19:14.586342 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.586347    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7lkx\" (UniqueName: \"kubernetes.io/projected/6d4884a1-4464-4f94-a336-d62ef88a777b-kube-api-access-f7lkx\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:19:14.927075 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.927035    2569 generic.go:356] "Generic (PLEG): container finished" podID="6d4884a1-4464-4f94-a336-d62ef88a777b" containerID="1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109" exitCode=0
Sep 05 08:19:14.927284 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.927121    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q"
Sep 05 08:19:14.927284 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.927123    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q" event={"ID":"6d4884a1-4464-4f94-a336-d62ef88a777b","Type":"ContainerDied","Data":"1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109"}
Sep 05 08:19:14.927284 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.927269    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb86cc4bd-2xm9q" event={"ID":"6d4884a1-4464-4f94-a336-d62ef88a777b","Type":"ContainerDied","Data":"e7f78949d1c56da7724077c6c8dae85dde640861fa280f0285ce6e1a27b0e8fc"}
Sep 05 08:19:14.927451 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.927299    2569 scope.go:117] "RemoveContainer" containerID="1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109"
Sep 05 08:19:14.938967 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.938944    2569 scope.go:117] "RemoveContainer" containerID="1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109"
Sep 05 08:19:14.939351 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:19:14.939319    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109\": container with ID starting with 1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109 not found: ID does not exist" containerID="1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109"
Sep 05 08:19:14.939472 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.939359    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109"} err="failed to get container status \"1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109\": rpc error: code = NotFound desc = could not find container \"1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109\": container with ID starting with 1ca605c12f15e64d76ca453a85c62a28fa67573a024e5a649244067b61047109 not found: ID does not exist"
Sep 05 08:19:14.954803 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.954756    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-2xm9q"]
Sep 05 08:19:14.957975 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:14.957949    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-2xm9q"]
Sep 05 08:19:15.570813 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.570781    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d4884a1-4464-4f94-a336-d62ef88a777b" path="/var/lib/kubelet/pods/6d4884a1-4464-4f94-a336-d62ef88a777b/volumes"
Sep 05 08:19:15.749127 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.749091    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-7g2lg"]
Sep 05 08:19:15.749559 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.749544    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d4884a1-4464-4f94-a336-d62ef88a777b" containerName="manager"
Sep 05 08:19:15.749559 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.749561    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d4884a1-4464-4f94-a336-d62ef88a777b" containerName="manager"
Sep 05 08:19:15.749660 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.749644    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="25866f6f-8128-42d2-b896-24e5e778a6d0" containerName="istio-proxy"
Sep 05 08:19:15.749660 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.749659    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d4884a1-4464-4f94-a336-d62ef88a777b" containerName="manager"
Sep 05 08:19:15.754293 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.754271    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:15.759080 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.758939    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 05 08:19:15.759080 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.758954    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9qpdw\""
Sep 05 08:19:15.759080 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.759040    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 05 08:19:15.759407 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.759203    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 05 08:19:15.762544 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.762504    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-7g2lg"]
Sep 05 08:19:15.899423 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.899333    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llkqt\" (UniqueName: \"kubernetes.io/projected/c3ffee71-ed85-4d7e-8b6e-74c36d678205-kube-api-access-llkqt\") pod \"maas-controller-7cb86cc4bd-7g2lg\" (UID: \"c3ffee71-ed85-4d7e-8b6e-74c36d678205\") " pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:15.899423 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.899396    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3ffee71-ed85-4d7e-8b6e-74c36d678205-metrics-certs\") pod \"maas-controller-7cb86cc4bd-7g2lg\" (UID: \"c3ffee71-ed85-4d7e-8b6e-74c36d678205\") " pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:15.899423 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:15.899422    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3ffee71-ed85-4d7e-8b6e-74c36d678205-webhook-cert\") pod \"maas-controller-7cb86cc4bd-7g2lg\" (UID: \"c3ffee71-ed85-4d7e-8b6e-74c36d678205\") " pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:16.000106 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.000053    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llkqt\" (UniqueName: \"kubernetes.io/projected/c3ffee71-ed85-4d7e-8b6e-74c36d678205-kube-api-access-llkqt\") pod \"maas-controller-7cb86cc4bd-7g2lg\" (UID: \"c3ffee71-ed85-4d7e-8b6e-74c36d678205\") " pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:16.000337 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.000139    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3ffee71-ed85-4d7e-8b6e-74c36d678205-metrics-certs\") pod \"maas-controller-7cb86cc4bd-7g2lg\" (UID: \"c3ffee71-ed85-4d7e-8b6e-74c36d678205\") " pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:16.000337 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.000165    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3ffee71-ed85-4d7e-8b6e-74c36d678205-webhook-cert\") pod \"maas-controller-7cb86cc4bd-7g2lg\" (UID: \"c3ffee71-ed85-4d7e-8b6e-74c36d678205\") " pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:16.002787 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.002758    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3ffee71-ed85-4d7e-8b6e-74c36d678205-metrics-certs\") pod \"maas-controller-7cb86cc4bd-7g2lg\" (UID: \"c3ffee71-ed85-4d7e-8b6e-74c36d678205\") " pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:16.002893 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.002788    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3ffee71-ed85-4d7e-8b6e-74c36d678205-webhook-cert\") pod \"maas-controller-7cb86cc4bd-7g2lg\" (UID: \"c3ffee71-ed85-4d7e-8b6e-74c36d678205\") " pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:16.007868 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.007844    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llkqt\" (UniqueName: \"kubernetes.io/projected/c3ffee71-ed85-4d7e-8b6e-74c36d678205-kube-api-access-llkqt\") pod \"maas-controller-7cb86cc4bd-7g2lg\" (UID: \"c3ffee71-ed85-4d7e-8b6e-74c36d678205\") " pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:16.066134 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.066103    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:16.339656 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.339623    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cb86cc4bd-7g2lg"]
Sep 05 08:19:16.342976 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:19:16.342950    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ffee71_ed85_4d7e_8b6e_74c36d678205.slice/crio-8dfdaf587b37a94fb434ed3ac02ad7bc0979be533c62c6b004bc156c00d18bf6 WatchSource:0}: Error finding container 8dfdaf587b37a94fb434ed3ac02ad7bc0979be533c62c6b004bc156c00d18bf6: Status 404 returned error can't find the container with id 8dfdaf587b37a94fb434ed3ac02ad7bc0979be533c62c6b004bc156c00d18bf6
Sep 05 08:19:16.947139 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.947039    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg" event={"ID":"c3ffee71-ed85-4d7e-8b6e-74c36d678205","Type":"ContainerStarted","Data":"d6cb243772de971412caadc104941b705718b4b40c39a73269fb8877d530b5cc"}
Sep 05 08:19:16.947139 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.947094    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg" event={"ID":"c3ffee71-ed85-4d7e-8b6e-74c36d678205","Type":"ContainerStarted","Data":"8dfdaf587b37a94fb434ed3ac02ad7bc0979be533c62c6b004bc156c00d18bf6"}
Sep 05 08:19:16.947598 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.947145    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:19:16.967477 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:16.967426    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg" podStartSLOduration=1.619559127 podStartE2EDuration="1.967409168s" totalImagesPullingTime="347.850041ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:19:16.96496409 +0000 UTC m=+1837.910285895" watchObservedRunningTime="2026-09-05 08:19:16.967409168 +0000 UTC m=+1837.912730972"
Sep 05 08:19:27.955824 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:19:27.955788    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cb86cc4bd-7g2lg"
Sep 05 08:23:13.398457 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.398421    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"]
Sep 05 08:23:13.402396 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.402377    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.405339 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.405315    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c62fz\""
Sep 05 08:23:13.411278 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.411126    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"]
Sep 05 08:23:13.593077 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.593038    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85640e73-6c02-4d52-857c-d4fc7ef1cbed-openshift-service-ca\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.593077 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.593077    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9w2x\" (UniqueName: \"kubernetes.io/projected/85640e73-6c02-4d52-857c-d4fc7ef1cbed-kube-api-access-k9w2x\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.593284 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.593100    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85640e73-6c02-4d52-857c-d4fc7ef1cbed-maas-api-tls\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.593284 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.593162    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85640e73-6c02-4d52-857c-d4fc7ef1cbed-metrics-certs\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.693904 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.693812    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85640e73-6c02-4d52-857c-d4fc7ef1cbed-maas-api-tls\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.693904 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.693856    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85640e73-6c02-4d52-857c-d4fc7ef1cbed-metrics-certs\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.694135 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.694036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85640e73-6c02-4d52-857c-d4fc7ef1cbed-openshift-service-ca\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.694135 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.694087    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9w2x\" (UniqueName: \"kubernetes.io/projected/85640e73-6c02-4d52-857c-d4fc7ef1cbed-kube-api-access-k9w2x\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.694954 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.694929    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85640e73-6c02-4d52-857c-d4fc7ef1cbed-openshift-service-ca\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.696501 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.696479    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85640e73-6c02-4d52-857c-d4fc7ef1cbed-maas-api-tls\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.696501 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.696493    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85640e73-6c02-4d52-857c-d4fc7ef1cbed-metrics-certs\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.702001 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.701983    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9w2x\" (UniqueName: \"kubernetes.io/projected/85640e73-6c02-4d52-857c-d4fc7ef1cbed-kube-api-access-k9w2x\") pod \"maas-api-7895c5cf96-qbg2j\" (UID: \"85640e73-6c02-4d52-857c-d4fc7ef1cbed\") " pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.715083 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.715066    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:13.986607 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:13.986572    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"]
Sep 05 08:23:13.988124 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:23:13.988094    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85640e73_6c02_4d52_857c_d4fc7ef1cbed.slice/crio-09f11026144073be0f28480bac9d5642e4554d5b8de6d7ed827563de3aeb316b WatchSource:0}: Error finding container 09f11026144073be0f28480bac9d5642e4554d5b8de6d7ed827563de3aeb316b: Status 404 returned error can't find the container with id 09f11026144073be0f28480bac9d5642e4554d5b8de6d7ed827563de3aeb316b
Sep 05 08:23:14.191774 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:14.191731    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j" event={"ID":"85640e73-6c02-4d52-857c-d4fc7ef1cbed","Type":"ContainerStarted","Data":"09f11026144073be0f28480bac9d5642e4554d5b8de6d7ed827563de3aeb316b"}
Sep 05 08:23:15.198798 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:15.198760    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j" event={"ID":"85640e73-6c02-4d52-857c-d4fc7ef1cbed","Type":"ContainerStarted","Data":"f6064972ade8af528fff00128fe21f989e8448eaf5b7fc6fab03beac455695ab"}
Sep 05 08:23:15.199164 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:15.198821    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:15.217172 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:15.217121    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j" podStartSLOduration=1.908140721 podStartE2EDuration="2.217106576s" totalImagesPullingTime="308.965855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 08:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:23:15.215658823 +0000 UTC m=+2076.160980629" watchObservedRunningTime="2026-09-05 08:23:15.217106576 +0000 UTC m=+2076.162428381"
Sep 05 08:23:21.211131 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:21.211103    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7895c5cf96-qbg2j"
Sep 05 08:23:21.260287 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:21.260257    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7548c976dd-msncz"]
Sep 05 08:23:21.260512 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:21.260485    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz" podUID="40c5a8b8-1aa9-4c59-b539-1d7db63d3515" containerName="maas-api" containerID="cri-o://1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c" gracePeriod=30
Sep 05 08:23:21.280041 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:21.279941    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz" podUID="40c5a8b8-1aa9-4c59-b539-1d7db63d3515" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.42:8443/health\": dial tcp 10.134.0.42:8443: connect: connection refused"
Sep 05 08:23:22.513668 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.513640    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 08:23:22.577288 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.577256    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-metrics-certs\") pod \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") "
Sep 05 08:23:22.577451 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.577318    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-openshift-service-ca\") pod \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") "
Sep 05 08:23:22.577451 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.577339    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2gcv\" (UniqueName: \"kubernetes.io/projected/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-kube-api-access-x2gcv\") pod \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") "
Sep 05 08:23:22.577451 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.577378    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-maas-api-tls\") pod \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\" (UID: \"40c5a8b8-1aa9-4c59-b539-1d7db63d3515\") "
Sep 05 08:23:22.577781 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.577753    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40c5a8b8-1aa9-4c59-b539-1d7db63d3515" (UID: "40c5a8b8-1aa9-4c59-b539-1d7db63d3515"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 08:23:22.579565 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.579540    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "40c5a8b8-1aa9-4c59-b539-1d7db63d3515" (UID: "40c5a8b8-1aa9-4c59-b539-1d7db63d3515"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:23:22.580118 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.580088    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40c5a8b8-1aa9-4c59-b539-1d7db63d3515" (UID: "40c5a8b8-1aa9-4c59-b539-1d7db63d3515"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 08:23:22.580118 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.580093    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-kube-api-access-x2gcv" (OuterVolumeSpecName: "kube-api-access-x2gcv") pod "40c5a8b8-1aa9-4c59-b539-1d7db63d3515" (UID: "40c5a8b8-1aa9-4c59-b539-1d7db63d3515"). InnerVolumeSpecName "kube-api-access-x2gcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 08:23:22.678783 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.678708    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-metrics-certs\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:23:22.678783 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.678737    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-openshift-service-ca\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:23:22.678783 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.678750    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2gcv\" (UniqueName: \"kubernetes.io/projected/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-kube-api-access-x2gcv\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:23:22.678783 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:22.678764    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c5a8b8-1aa9-4c59-b539-1d7db63d3515-maas-api-tls\") on node \"ip-10-0-140-64.ec2.internal\" DevicePath \"\""
Sep 05 08:23:23.251128 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:23.251091    2569 generic.go:356] "Generic (PLEG): container finished" podID="40c5a8b8-1aa9-4c59-b539-1d7db63d3515" containerID="1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c" exitCode=0
Sep 05 08:23:23.251340 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:23.251136    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz"
Sep 05 08:23:23.251340 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:23.251138    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz" event={"ID":"40c5a8b8-1aa9-4c59-b539-1d7db63d3515","Type":"ContainerDied","Data":"1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c"}
Sep 05 08:23:23.251340 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:23.251263    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7548c976dd-msncz" event={"ID":"40c5a8b8-1aa9-4c59-b539-1d7db63d3515","Type":"ContainerDied","Data":"23d4fc67bc3a002df04dfceded7077a697fda6e09d2f1daccd308d0157420f0d"}
Sep 05 08:23:23.251340 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:23.251291    2569 scope.go:117] "RemoveContainer" containerID="1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c"
Sep 05 08:23:23.262373 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:23.262349    2569 scope.go:117] "RemoveContainer" containerID="1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c"
Sep 05 08:23:23.262725 ip-10-0-140-64 kubenswrapper[2569]: E0905 08:23:23.262706    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c\": container with ID starting with 1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c not found: ID does not exist" containerID="1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c"
Sep 05 08:23:23.262796 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:23.262738    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c"} err="failed to get container status \"1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c\": rpc error: code = NotFound desc = could not find container \"1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c\": container with ID starting with 1e2138efbb5e2b1774df2b29986a55984f0e0c62de6a6050550163f766a8ab2c not found: ID does not exist"
Sep 05 08:23:23.273830 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:23.273804    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7548c976dd-msncz"]
Sep 05 08:23:23.275693 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:23.275668    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7548c976dd-msncz"]
Sep 05 08:23:23.571834 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:23.571800    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c5a8b8-1aa9-4c59-b539-1d7db63d3515" path="/var/lib/kubelet/pods/40c5a8b8-1aa9-4c59-b539-1d7db63d3515/volumes"
Sep 05 08:23:39.738350 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:39.738325    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:23:39.747135 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:39.747114    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xp7vx_81e9ce3b-89e2-488c-b4f3-4d4d4c2726ae/ovn-acl-logging/0.log"
Sep 05 08:23:44.515136 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:44.515099    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zw6gz_c16f8ac7-d8ed-4834-b3d0-c582cee5d746/manager/0.log"
Sep 05 08:23:44.632232 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:44.632196    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tsc29_6189754b-abc7-4675-a676-6bb1babd4599/manager/0.log"
Sep 05 08:23:44.753400 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:44.753362    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7cb86cc4bd-7g2lg_c3ffee71-ed85-4d7e-8b6e-74c36d678205/manager/0.log"
Sep 05 08:23:45.337170 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:45.337138    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc4f56d7-rf9rp_b0f0e25c-ce27-4b81-ad2a-0ee1622992c9/copy-manifests/0.log"
Sep 05 08:23:45.348865 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:45.348842    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc4f56d7-rf9rp_b0f0e25c-ce27-4b81-ad2a-0ee1622992c9/manager/0.log"
Sep 05 08:23:45.802388 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:45.802315    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7895c5cf96-qbg2j_85640e73-6c02-4d52-857c-d4fc7ef1cbed/maas-api/0.log"
Sep 05 08:23:46.157744 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:46.157697    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-sd9fl_93ccecc9-4561-4de6-bf12-f2e0b900e447/postgres/0.log"
Sep 05 08:23:47.675154 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:47.675126    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-rl65f_ecabc342-f49c-4e5a-a6b2-663d2d60bf8b/manager/0.log"
Sep 05 08:23:48.490909 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:48.490875    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-bzs86_2b378fb5-4133-490b-af8f-065518cc9b0a/istio-proxy/1.log"
Sep 05 08:23:48.635026 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:48.634994    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lvpvq_34d3e8dc-f864-499f-bd75-097835e087ad/payload-pre-processing/0.log"
Sep 05 08:23:48.785127 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:48.785051    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-fxvvn_31f840d8-c593-439e-9144-d2004574c575/payload-processing/0.log"
Sep 05 08:23:49.274025 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:49.273993    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-pbh2b_8c44b3ff-907f-422e-bf0d-b03efb882c7a/main/0.log"
Sep 05 08:23:49.601064 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:49.601037    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-nd85f_4f70be3c-1197-491b-9e67-acb8468f288f/main/0.log"
Sep 05 08:23:59.048460 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:59.048422    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6nw26_3bb3f562-7acc-4fcd-8f35-b4b8e18e2a5a/global-pull-secret-syncer/0.log"
Sep 05 08:23:59.206183 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:59.206153    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v2j86_a07aa982-f058-45d8-b69d-f6f248285657/konnectivity-agent/0.log"
Sep 05 08:23:59.283565 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:23:59.283513    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-64.ec2.internal_1120dc1be11d71eb0105a790715e7795/haproxy/0.log"
Sep 05 08:24:04.378141 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:04.378113    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-rl65f_ecabc342-f49c-4e5a-a6b2-663d2d60bf8b/manager/0.log"
Sep 05 08:24:05.817631 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:05.817602    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20073340-5926-4f77-b1ff-f70558d61bb2/alertmanager/0.log"
Sep 05 08:24:05.839343 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:05.839322    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20073340-5926-4f77-b1ff-f70558d61bb2/config-reloader/0.log"
Sep 05 08:24:05.868007 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:05.867987    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20073340-5926-4f77-b1ff-f70558d61bb2/kube-rbac-proxy-web/0.log"
Sep 05 08:24:05.892844 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:05.892825    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20073340-5926-4f77-b1ff-f70558d61bb2/kube-rbac-proxy/0.log"
Sep 05 08:24:05.914637 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:05.914617    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20073340-5926-4f77-b1ff-f70558d61bb2/kube-rbac-proxy-metric/0.log"
Sep 05 08:24:05.935501 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:05.935446    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20073340-5926-4f77-b1ff-f70558d61bb2/prom-label-proxy/0.log"
Sep 05 08:24:05.956870 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:05.956852    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20073340-5926-4f77-b1ff-f70558d61bb2/init-config-reloader/0.log"
Sep 05 08:24:06.027730 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.027704    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hhgqh_de5cc2c8-7bd9-4706-8788-8b8479a7087a/kube-state-metrics/0.log"
Sep 05 08:24:06.047274 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.047250    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hhgqh_de5cc2c8-7bd9-4706-8788-8b8479a7087a/kube-rbac-proxy-main/0.log"
Sep 05 08:24:06.068756 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.068733    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hhgqh_de5cc2c8-7bd9-4706-8788-8b8479a7087a/kube-rbac-proxy-self/0.log"
Sep 05 08:24:06.105465 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.105448    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-769d67686c-7gbxm_507dfce2-0001-4920-af9e-21af592e5a3c/metrics-server/0.log"
Sep 05 08:24:06.143314 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.143275    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-f2nwv_db7e756b-443f-469c-880a-8215b6ac72e3/monitoring-plugin/0.log"
Sep 05 08:24:06.256923 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.256857    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w524v_bab5ab31-4965-420e-a2e2-7c53dcd98c85/node-exporter/0.log"
Sep 05 08:24:06.277267 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.277246    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w524v_bab5ab31-4965-420e-a2e2-7c53dcd98c85/kube-rbac-proxy/0.log"
Sep 05 08:24:06.297216 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.297199    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w524v_bab5ab31-4965-420e-a2e2-7c53dcd98c85/init-textfile/0.log"
Sep 05 08:24:06.410589 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.410561    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-67csw_8329852e-bd3d-453d-b2ee-ae97d1b6f8cd/kube-rbac-proxy-main/0.log"
Sep 05 08:24:06.439691 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.439670    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-67csw_8329852e-bd3d-453d-b2ee-ae97d1b6f8cd/kube-rbac-proxy-self/0.log"
Sep 05 08:24:06.462976 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.462956    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-67csw_8329852e-bd3d-453d-b2ee-ae97d1b6f8cd/openshift-state-metrics/0.log"
Sep 05 08:24:06.521147 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.521087    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa167a5-c208-424e-acc7-9618be9cf172/prometheus/0.log"
Sep 05 08:24:06.550211 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.550194    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa167a5-c208-424e-acc7-9618be9cf172/config-reloader/0.log"
Sep 05 08:24:06.584974 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.584953    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa167a5-c208-424e-acc7-9618be9cf172/thanos-sidecar/0.log"
Sep 05 08:24:06.618889 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.618866    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa167a5-c208-424e-acc7-9618be9cf172/kube-rbac-proxy-web/0.log"
Sep 05 08:24:06.649242 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.649225    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa167a5-c208-424e-acc7-9618be9cf172/kube-rbac-proxy/0.log"
Sep 05 08:24:06.682731 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.682710    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa167a5-c208-424e-acc7-9618be9cf172/kube-rbac-proxy-thanos/0.log"
Sep 05 08:24:06.703582 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.703558    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eaa167a5-c208-424e-acc7-9618be9cf172/init-config-reloader/0.log"
Sep 05 08:24:06.734902 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.734880    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-jpf6c_0764a774-f924-49d5-ac9a-c006a2aeae64/prometheus-operator/0.log"
Sep 05 08:24:06.757897 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.757877    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-jpf6c_0764a774-f924-49d5-ac9a-c006a2aeae64/kube-rbac-proxy/0.log"
Sep 05 08:24:06.787474 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.787419    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-t2tsg_5f5d1728-d25d-41ba-9279-6d688077e828/prometheus-operator-admission-webhook/0.log"
Sep 05 08:24:06.851066 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.851043    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-596f58f6d6-k92n5_9cf03b4c-c779-49d0-be96-54963187bbb6/telemeter-client/0.log"
Sep 05 08:24:06.920693 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.920670    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-596f58f6d6-k92n5_9cf03b4c-c779-49d0-be96-54963187bbb6/reload/0.log"
Sep 05 08:24:06.966206 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:06.966177    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-596f58f6d6-k92n5_9cf03b4c-c779-49d0-be96-54963187bbb6/kube-rbac-proxy/0.log"
Sep 05 08:24:07.007313 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:07.007281    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8664b866fc-qlcph_a7277c30-a1dd-4fbf-b05c-421072627753/thanos-query/0.log"
Sep 05 08:24:07.051158 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:07.051096    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8664b866fc-qlcph_a7277c30-a1dd-4fbf-b05c-421072627753/kube-rbac-proxy-web/0.log"
Sep 05 08:24:07.093411 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:07.093388    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8664b866fc-qlcph_a7277c30-a1dd-4fbf-b05c-421072627753/kube-rbac-proxy/0.log"
Sep 05 08:24:07.135902 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:07.135882    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8664b866fc-qlcph_a7277c30-a1dd-4fbf-b05c-421072627753/prom-label-proxy/0.log"
Sep 05 08:24:07.176098 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:07.176079    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8664b866fc-qlcph_a7277c30-a1dd-4fbf-b05c-421072627753/kube-rbac-proxy-rules/0.log"
Sep 05 08:24:07.218269 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:07.218247    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8664b866fc-qlcph_a7277c30-a1dd-4fbf-b05c-421072627753/kube-rbac-proxy-metrics/0.log"
Sep 05 08:24:09.346509 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.346473    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"]
Sep 05 08:24:09.347096 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.347076    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c5a8b8-1aa9-4c59-b539-1d7db63d3515" containerName="maas-api"
Sep 05 08:24:09.347150 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.347099    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c5a8b8-1aa9-4c59-b539-1d7db63d3515" containerName="maas-api"
Sep 05 08:24:09.347196 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.347187    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c5a8b8-1aa9-4c59-b539-1d7db63d3515" containerName="maas-api"
Sep 05 08:24:09.350662 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.350647    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.353340 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.353311    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-24lm5\"/\"openshift-service-ca.crt\""
Sep 05 08:24:09.353478 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.353416    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-24lm5\"/\"kube-root-ca.crt\""
Sep 05 08:24:09.354655 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.354638    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-24lm5\"/\"default-dockercfg-z7vr5\""
Sep 05 08:24:09.359487 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.359467    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"]
Sep 05 08:24:09.423220 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.423192    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-sys\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.423340 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.423245    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-lib-modules\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.423340 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.423273    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv7qf\" (UniqueName: \"kubernetes.io/projected/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-kube-api-access-tv7qf\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.423340 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.423306    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-podres\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.423459 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.423374    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-proc\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.524190 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.524157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-proc\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.524355 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.524200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-sys\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.524355 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.524258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-lib-modules\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.524355 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.524282    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv7qf\" (UniqueName: \"kubernetes.io/projected/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-kube-api-access-tv7qf\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.524355 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.524303    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-podres\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.524355 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.524303    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-proc\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.524628 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.524395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-sys\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.524628 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.524422    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-podres\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.524628 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.524423    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-lib-modules\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.531878 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.531855    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv7qf\" (UniqueName: \"kubernetes.io/projected/cb5e0f7a-e939-4d26-94ec-e5be0137a98f-kube-api-access-tv7qf\") pod \"perf-node-gather-daemonset-fwtmf\" (UID: \"cb5e0f7a-e939-4d26-94ec-e5be0137a98f\") " pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.661458 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.661395    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:09.702645 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.702618    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7bbfb68d4f-2dt4n_ac2faf70-6059-45c1-b9f0-57251f8e9ef3/console/0.log"
Sep 05 08:24:09.932813 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:09.930685    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"]
Sep 05 08:24:09.934864 ip-10-0-140-64 kubenswrapper[2569]: W0905 08:24:09.934837    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcb5e0f7a_e939_4d26_94ec_e5be0137a98f.slice/crio-94d3cec1254894370c53a1641f3a9f37376e0ff6bbce094c08e02359766f9d4f WatchSource:0}: Error finding container 94d3cec1254894370c53a1641f3a9f37376e0ff6bbce094c08e02359766f9d4f: Status 404 returned error can't find the container with id 94d3cec1254894370c53a1641f3a9f37376e0ff6bbce094c08e02359766f9d4f
Sep 05 08:24:10.496776 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:10.496742    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf" event={"ID":"cb5e0f7a-e939-4d26-94ec-e5be0137a98f","Type":"ContainerStarted","Data":"c0924c2316fbdec0006ab4ffb8c4e6511024e517c7565ed872a5e259f5677dc4"}
Sep 05 08:24:10.497268 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:10.496793    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf" event={"ID":"cb5e0f7a-e939-4d26-94ec-e5be0137a98f","Type":"ContainerStarted","Data":"94d3cec1254894370c53a1641f3a9f37376e0ff6bbce094c08e02359766f9d4f"}
Sep 05 08:24:10.497268 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:10.496888    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:10.518157 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:10.518105    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf" podStartSLOduration=1.518091029 podStartE2EDuration="1.518091029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 08:24:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 08:24:10.516271382 +0000 UTC m=+2131.461593169" watchObservedRunningTime="2026-09-05 08:24:10.518091029 +0000 UTC m=+2131.463412834"
Sep 05 08:24:11.623430 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:11.623403    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vtcdn_1a0f2752-8a69-451c-a57d-c38242e3350d/dns/0.log"
Sep 05 08:24:11.644407 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:11.644382    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vtcdn_1a0f2752-8a69-451c-a57d-c38242e3350d/kube-rbac-proxy/0.log"
Sep 05 08:24:11.716045 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:11.716019    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q754g_97e3d2a4-be3f-487f-b2a3-380e78fed6e1/dns-node-resolver/0.log"
Sep 05 08:24:12.318051 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:12.318024    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mgwk5_632a9615-66ff-47b8-b9e5-4357b516b064/node-ca/0.log"
Sep 05 08:24:13.685985 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:13.685938    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-bzs86_2b378fb5-4133-490b-af8f-065518cc9b0a/istio-proxy/0.log"
Sep 05 08:24:13.694334 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:13.694287    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-bzs86_2b378fb5-4133-490b-af8f-065518cc9b0a/istio-proxy/1.log"
Sep 05 08:24:13.727740 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:13.727716    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lvpvq_34d3e8dc-f864-499f-bd75-097835e087ad/payload-pre-processing/0.log"
Sep 05 08:24:13.760673 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:13.760650    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-fxvvn_31f840d8-c593-439e-9144-d2004574c575/payload-processing/0.log"
Sep 05 08:24:14.630098 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:14.630068    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ksjgr_735870a0-b9d2-4e05-8e45-7e01af14f5a3/serve-healthcheck-canary/0.log"
Sep 05 08:24:15.298284 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:15.298255    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ph7k8_bbfb7b6c-970f-4d09-8eaf-e4944b723741/kube-rbac-proxy/0.log"
Sep 05 08:24:15.318471 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:15.318447    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ph7k8_bbfb7b6c-970f-4d09-8eaf-e4944b723741/exporter/0.log"
Sep 05 08:24:15.339430 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:15.339396    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ph7k8_bbfb7b6c-970f-4d09-8eaf-e4944b723741/extractor/0.log"
Sep 05 08:24:16.512657 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:16.512632    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-24lm5/perf-node-gather-daemonset-fwtmf"
Sep 05 08:24:17.396043 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:17.396019    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zw6gz_c16f8ac7-d8ed-4834-b3d0-c582cee5d746/manager/0.log"
Sep 05 08:24:17.583294 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:17.583269    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tsc29_6189754b-abc7-4675-a676-6bb1babd4599/manager/0.log"
Sep 05 08:24:17.630919 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:17.630893    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7cb86cc4bd-7g2lg_c3ffee71-ed85-4d7e-8b6e-74c36d678205/manager/0.log"
Sep 05 08:24:17.911306 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:17.911272    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc4f56d7-rf9rp_b0f0e25c-ce27-4b81-ad2a-0ee1622992c9/manager/0.log"
Sep 05 08:24:17.931822 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:17.931799    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc4f56d7-rf9rp_b0f0e25c-ce27-4b81-ad2a-0ee1622992c9/copy-manifests/0.log"
Sep 05 08:24:19.366949 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:19.366919    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-bd655598-ndghx_70f0684c-52f2-43d0-ad16-8f358f2be4c4/manager/0.log"
Sep 05 08:24:20.703237 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:20.703200    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-bzs86_2b378fb5-4133-490b-af8f-065518cc9b0a/istio-proxy/0.log"
Sep 05 08:24:20.713981 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:20.713958    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-bzs86_2b378fb5-4133-490b-af8f-065518cc9b0a/istio-proxy/1.log"
Sep 05 08:24:20.736872 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:20.736851    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lvpvq_34d3e8dc-f864-499f-bd75-097835e087ad/payload-pre-processing/0.log"
Sep 05 08:24:20.760880 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:20.760859    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-fxvvn_31f840d8-c593-439e-9144-d2004574c575/payload-processing/0.log"
Sep 05 08:24:22.711078 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:22.711043    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-bzs86_2b378fb5-4133-490b-af8f-065518cc9b0a/istio-proxy/0.log"
Sep 05 08:24:22.719389 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:22.719363    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-bzs86_2b378fb5-4133-490b-af8f-065518cc9b0a/istio-proxy/1.log"
Sep 05 08:24:22.747783 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:22.747757    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lvpvq_34d3e8dc-f864-499f-bd75-097835e087ad/payload-pre-processing/0.log"
Sep 05 08:24:22.773158 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:22.773136    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-fxvvn_31f840d8-c593-439e-9144-d2004574c575/payload-processing/0.log"
Sep 05 08:24:25.984949 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:25.984917    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9tqz9_4acfb359-d4f2-4e0a-a442-7871d3000d97/kube-multus/0.log"
Sep 05 08:24:26.334179 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:26.334154    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvv55_6eeab356-b5db-4ccf-920e-3bc31edc9962/kube-multus-additional-cni-plugins/0.log"
Sep 05 08:24:26.357278 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:26.357252    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvv55_6eeab356-b5db-4ccf-920e-3bc31edc9962/egress-router-binary-copy/0.log"
Sep 05 08:24:26.378846 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:26.378822    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvv55_6eeab356-b5db-4ccf-920e-3bc31edc9962/cni-plugins/0.log"
Sep 05 08:24:26.399893 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:26.399873    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvv55_6eeab356-b5db-4ccf-920e-3bc31edc9962/bond-cni-plugin/0.log"
Sep 05 08:24:26.424129 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:26.424110    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvv55_6eeab356-b5db-4ccf-920e-3bc31edc9962/routeoverride-cni/0.log"
Sep 05 08:24:26.449156 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:26.449138    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvv55_6eeab356-b5db-4ccf-920e-3bc31edc9962/whereabouts-cni-bincopy/0.log"
Sep 05 08:24:26.470394 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:26.470379    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mvv55_6eeab356-b5db-4ccf-920e-3bc31edc9962/whereabouts-cni/0.log"
Sep 05 08:24:26.579490 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:26.579465    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bcxfm_6264509b-4382-45b2-a6f0-d8af26dd57f1/network-metrics-daemon/0.log"
Sep 05 08:24:26.597842 ip-10-0-140-64 kubenswrapper[2569]: I0905 08:24:26.597775    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bcxfm_6264509b-4382-45b2-a6f0-d8af26dd57f1/kube-rbac-proxy/0.log"