Sep 09 12:10:55.252022 ip-10-0-131-207 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 12:10:55.681731 ip-10-0-131-207 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:10:55.681731 ip-10-0-131-207 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 12:10:55.681731 ip-10-0-131-207 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:10:55.681731 ip-10-0-131-207 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 12:10:55.681731 ip-10-0-131-207 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:10:55.682401 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.681813    2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 12:10:55.688687 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688660    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:10:55.688687 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688683    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:10:55.688687 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688687    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:10:55.688687 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688690    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:10:55.688687 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688694    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688711    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688716    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688720    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688722    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688725    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688728    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688730    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688733    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688735    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688738    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688741    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688743    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688746    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688749    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688751    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688754    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688756    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688759    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688762    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:10:55.688911 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688764    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688767    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688771    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688773    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688776    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688778    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688784    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688787    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688789    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688792    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688796    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688800    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688803    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688806    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688811    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688813    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688816    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688818    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688821    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:10:55.689391 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688824    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688826    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688829    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688831    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688834    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688837    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688839    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688842    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688845    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688847    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688850    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688852    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688855    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688857    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688860    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688864    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688866    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688869    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688871    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:10:55.689885 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688875    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688879    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688882    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688885    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688888    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688891    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688894    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688897    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688902    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688905    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688908    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688910    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688913    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688916    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688918    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688921    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688923    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688926    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688928    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688931    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:10:55.690342 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688933    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688937    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688939    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688942    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688944    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688947    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688950    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688952    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688954    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688957    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688960    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688962    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688965    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688967    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688970    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688972    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.688975    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690522    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690530    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690534    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:10:55.690836 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690537    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690540    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690543    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690545    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690548    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690551    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690553    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690556    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690558    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690560    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690563    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690566    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690568    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690571    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690575    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690578    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690580    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690583    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690586    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690589    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:10:55.691299 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690592    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690594    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690597    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690600    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690602    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690605    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690607    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690610    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690613    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690615    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690619    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690623    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690626    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690629    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690632    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690634    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690637    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690640    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690642    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:10:55.691827 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690644    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690647    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690649    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690652    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690654    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690656    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690659    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690662    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690666    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690670    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690672    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690675    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690678    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690681    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690684    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690687    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690689    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690693    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690716    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690719    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:10:55.692294 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690722    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690725    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690727    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690730    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690733    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690735    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690738    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690740    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690743    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690745    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690748    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690750    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690753    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690755    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690757    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690760    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690763    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690765    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690768    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690771    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:10:55.692904 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690773    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690776    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690779    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690782    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690784    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690787    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690789    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690792    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690794    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690797    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690800    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690804    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690807    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690809    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690812    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690814    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.690816    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691209    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691213    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691216    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:10:55.693404 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691218    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691221    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691224    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691227    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691230    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691232    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691235    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691238    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691242    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691245    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691248    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691251    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691253    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691257    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691259    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691262    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691264    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691266    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691269    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:10:55.693950 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691272    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691274    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691276    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691279    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691281    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691284    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691286    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691289    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691291    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691294    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691296    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691299    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691302    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691304    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691306    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691309    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691311    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691314    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691317    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691319    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:10:55.694416 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691321    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691324    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691327    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691329    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691332    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691335    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691337    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691340    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691343    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691346    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691348    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691351    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691353    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691355    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691358    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691360    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691363    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691367    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691371    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:10:55.694935 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691373    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691376    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691379    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691382    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691384    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691387    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691391    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691394    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691396    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691400    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691403    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691406    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691408    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691411    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691413    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691416    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691419    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691421    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691424    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:10:55.695426 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691426    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691429    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691432    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691434    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691437    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691439    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691441    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691444    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691446    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691448    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691451    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691453    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691456    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691458    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691461    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691463    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691466    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691468    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.691470    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691547    2585 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691578    2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 12:10:55.695886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691585    2585 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691590    2585 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691593    2585 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691597    2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691601    2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691605    2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691608    2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691611    2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691615    2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691618    2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691621    2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691624    2585 flags.go:64] FLAG: --cgroup-root=""
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691627    2585 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691629    2585 flags.go:64] FLAG: --client-ca-file=""
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691633    2585 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691636    2585 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691640    2585 flags.go:64] FLAG: --cluster-domain=""
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691642    2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691645    2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691648    2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691651    2585 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691655    2585 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691658    2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 12:10:55.696386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691660    2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691663    2585 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691666    2585 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691669    2585 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691672    2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691675    2585 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691677    2585 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691683    2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691686    2585 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691689    2585 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691692    2585 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691709    2585 flags.go:64] FLAG: --enable-server="true"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691712    2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691717    2585 flags.go:64] FLAG: --event-burst="100"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691721    2585 flags.go:64] FLAG: --event-qps="50"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691724    2585 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691727    2585 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691730    2585 flags.go:64] FLAG: --eviction-hard=""
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691736    2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691739    2585 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691742    2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691745    2585 flags.go:64] FLAG: --eviction-soft=""
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691748    2585 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691751    2585 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691754    2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 12:10:55.696994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691757    2585 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691760    2585 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691763    2585 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691765    2585 flags.go:64] FLAG: --feature-gates=""
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691770    2585 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691773    2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691775    2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691778    2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691781    2585 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691784    2585 flags.go:64] FLAG: --help="false"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691787    2585 flags.go:64] FLAG: --hostname-override="ip-10-0-131-207.ec2.internal"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691790    2585 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691792    2585 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691795    2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691798    2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691802    2585 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691804    2585 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691807    2585 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691810    2585 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691812    2585 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691815    2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691818    2585 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691821    2585 flags.go:64] FLAG: --kube-reserved=""
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691825    2585 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 12:10:55.697622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691828    2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691833    2585 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691836    2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691838    2585 flags.go:64] FLAG: --lock-file=""
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691841    2585 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691844    2585 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691847    2585 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691852    2585 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691856    2585 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691859    2585 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691862    2585 flags.go:64] FLAG: --logging-format="text"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691865    2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691868    2585 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691871    2585 flags.go:64] FLAG: --manifest-url=""
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691874    2585 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691878    2585 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691881    2585 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691885    2585 flags.go:64] FLAG: --max-pods="110"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691888    2585 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691891    2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691893    2585 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691896    2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691899    2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691902    2585 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691905    2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 12:10:55.698243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691913    2585 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691916    2585 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691919    2585 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691922    2585 flags.go:64] FLAG: --pod-cidr=""
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691925    2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691930    2585 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691934    2585 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691937    2585 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691939    2585 flags.go:64] FLAG: --port="10250"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691944    2585 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691947    2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0beca2d38616ff709"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691950    2585 flags.go:64] FLAG: --qos-reserved=""
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691953    2585 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691956    2585 flags.go:64] FLAG: --register-node="true"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691959    2585 flags.go:64] FLAG: --register-with-taints=""
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691963    2585 flags.go:64] FLAG: --registry-burst="10"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691966    2585 flags.go:64] FLAG: --registry-qps="5"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691969    2585 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691972    2585 flags.go:64] FLAG: --reserved-memory=""
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691976    2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691978    2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691981    2585 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691984    2585 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691987    2585 flags.go:64] FLAG: --runonce="false"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691990    2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 12:10:55.698865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691993    2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691996    2585 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.691999    2585 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692002    2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692005    2585 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692008    2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692011    2585 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692014    2585 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692017    2585 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692019    2585 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692022    2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692025    2585 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692028    2585 flags.go:64] FLAG: --system-cgroups=""
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692032    2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692037    2585 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692039    2585 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692043    2585 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692047    2585 flags.go:64] FLAG: --tls-min-version=""
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692050    2585 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692053    2585 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692055    2585 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692058    2585 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692061    2585 flags.go:64] FLAG: --v="2"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692065    2585 flags.go:64] FLAG: --version="false"
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692069    2585 flags.go:64] FLAG: --vmodule=""
Sep 09 12:10:55.699469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692073    2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.692076    2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692195    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692199    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692203    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692206    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692209    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692211    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692214    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692216    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692219    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692221    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692224    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692226    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692229    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692231    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692234    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692236    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692239    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692242    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:10:55.700109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692246    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692248    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692251    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692255    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692257    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692260    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692262    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692265    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692267    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692270    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692272    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692274    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692277    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692280    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692282    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692285    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692287    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692289    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692292    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692295    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:10:55.700640 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692297    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692300    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692303    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692306    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692325    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692329    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692332    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692335    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692339    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692341    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692344    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692347    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692350    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692353    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692355    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692359    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692361    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692364    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692366    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692369    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:10:55.701142 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692371    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692374    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692377    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692379    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692381    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692384    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692387    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692390    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692393    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692395    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692397    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692400    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692402    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692404    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692407    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692409    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692413    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692416    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692420    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:10:55.701629 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692423    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692426    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692428    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692431    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692434    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692437    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692439    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692442    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692446    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692448    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692451    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692453    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692455    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692458    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692461    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692463    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692466    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692468    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692470    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692473    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:10:55.702127 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692476    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.692478    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.693149    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.701118    2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.701134    2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.702156    2585 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.702167    2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702224    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702229    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702233    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702236    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702239    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702242    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702245    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702247    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:10:55.702607 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702251    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702253    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702256    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702259    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702261    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702264    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702267    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702269    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702272    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702274    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702277    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702279    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702281    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702284    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702288    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702291    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702294    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702296    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702299    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:10:55.703011 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702301    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702304    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702308    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702310    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702313    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702317    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702320    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702324    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702326    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702329    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702332    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702334    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702337    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702339    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702341    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702344    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702346    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702349    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702351    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:10:55.703461 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702354    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702356    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702359    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702361    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702364    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702366    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702369    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702372    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702374    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702377    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702380    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702383    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702385    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702388    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702390    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702393    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702400    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702402    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702405    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702408    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:10:55.703974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702410    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702412    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702415    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702417    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702420    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702423    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702425    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702427    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702430    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702432    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702435    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702437    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702440    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702442    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702445    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702447    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702450    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702452    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702454    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702457    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:10:55.704456 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702459    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702462    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702465    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702467    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702470    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702472    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702475    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702478    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702481    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702485    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702487    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702490    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702493    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.702498    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702629    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:10:55.704949 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702635    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702638    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702640    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702643    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702645    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702648    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702650    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702653    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702656    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702658    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702661    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702663    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702666    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702668    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702670    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702673    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702676    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702678    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702681    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702683    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:10:55.705333 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702686    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702689    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702692    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702709    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702713    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702717    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702721    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702725    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702728    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702730    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702733    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702735    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702738    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702740    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702743    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702746    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702749    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702751    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702754    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702756    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:10:55.705845 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702758    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702761    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702763    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702766    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702768    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702771    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702773    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702776    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702779    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702781    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702784    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702786    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702789    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702791    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702794    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702796    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702798    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702801    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702803    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:10:55.706336 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702806    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702810    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702814    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702818    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702821    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702824    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702827    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702829    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702832    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702834    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702837    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702839    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702842    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702844    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702848    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702851    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702853    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702856    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702858    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:10:55.706895 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702861    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702863    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702866    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702868    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702871    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702874    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702876    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702879    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702881    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702884    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702886    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702888    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702891    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702893    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702896    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702898    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702901    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702903    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702906    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:10:55.707350 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:55.702908    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:10:55.707817 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.702912    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:10:55.707817 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.703024    2585 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 12:10:55.707817 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.705431    2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 12:10:55.707817 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.706612    2585 server.go:1019] "Starting client certificate rotation"
Sep 09 12:10:55.707817 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.706730    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:10:55.707817 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.706761    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:10:55.709428 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.709415    2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:10:55.710595 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.710578    2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:10:55.725984 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.725838    2585 log.go:25] "Validated CRI v1 runtime API"
Sep 09 12:10:55.729339 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.729321    2585 log.go:25] "Validated CRI v1 image API"
Sep 09 12:10:55.731151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.731136    2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 12:10:55.737005 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.736984    2585 fs.go:135] Filesystem UUIDs: map[13f1dda9-5564-4c15-baf9-ca348f1706b6:/dev/nvme0n1p4 63317f7e-a9b8-4401-b7ab-f8b55ce42d29:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 09 12:10:55.737005 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.737003    2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 12:10:55.738523 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.738508    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:10:55.742901 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.742790    2585 manager.go:216] Machine: {Timestamp:2026-09-09 12:10:55.742258273 +0000 UTC m=+0.383877470 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3113266 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2331089cb407bc1b1030bd34861564 SystemUUID:ec233108-9cb4-07bc-1b10-30bd34861564 BootID:e4995cdb-0132-4770-b719-006929aac856 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b4:64:09:36:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b4:64:09:36:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:53:c9:5b:ac:04 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 12:10:55.742901 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.742896    2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 12:10:55.743008 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.742979    2585 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 12:10:55.745881 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.745853    2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 12:10:55.746025 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.745884    2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-207.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 12:10:55.746777 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.746768    2585 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 12:10:55.746812 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.746782    2585 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 12:10:55.746812 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.746790    2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:10:55.747823 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.747813    2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:10:55.749707 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.749684    2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 12:10:55.754541 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.754530    2585 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:10:55.754648 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.754639    2585 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 12:10:55.754677 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.754670    2585 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 12:10:55.754721 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.754680    2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 12:10:55.754721 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.754690    2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 12:10:55.754721 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.754713    2585 kubelet.go:398] "Adding apiserver pod source"
Sep 09 12:10:55.754721 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.754722    2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 12:10:55.755079 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.755068    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:10:55.755120 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.755089    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:10:55.758340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.758323    2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 12:10:55.761207 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.761194    2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 12:10:55.761255 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.761226    2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 12:10:55.761392 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.761383    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 12:10:55.761424 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.761398    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 12:10:55.761424 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.761405    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 12:10:55.761424 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.761410    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 12:10:55.761424 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.761420    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 12:10:55.761524 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.761429    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 12:10:55.762091 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.762080    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 12:10:55.762122 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.762093    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 12:10:55.762122 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.762101    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 12:10:55.762122 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.762107    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 12:10:55.762122 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.762120    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 12:10:55.762224 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.762128    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 12:10:55.763891 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.763881    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 12:10:55.763891 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.763891    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 12:10:55.768066 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.768040    2585 server.go:1268] "Started kubelet"
Sep 09 12:10:55.768164 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.768126    2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 12:10:55.768235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.768181    2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-207.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 12:10:55.768294 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.768275    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 12:10:55.768294 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.768275    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-207.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 12:10:55.768400 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.768269    2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 12:10:55.768400 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.768339    2585 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 12:10:55.769090 ip-10-0-131-207 systemd[1]: Started Kubernetes Kubelet.
Sep 09 12:10:55.770181 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.770041    2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 12:10:55.775719 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.775027    2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 12:10:55.775719 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.775195    2585 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 12:10:55.779746 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.779725    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 12:10:55.780111 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.778947    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-207.ec2.internal.18d3a5df5de82fae  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-207.ec2.internal,UID:ip-10-0-131-207.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-207.ec2.internal,},FirstTimestamp:2026-09-09 12:10:55.768006574 +0000 UTC m=+0.409625769,LastTimestamp:2026-09-09 12:10:55.768006574 +0000 UTC m=+0.409625769,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-207.ec2.internal,}"
Sep 09 12:10:55.780370 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.780356    2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 12:10:55.780551 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.780532    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-207.ec2.internal\" not found"
Sep 09 12:10:55.781138 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.781122    2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 12:10:55.781560 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.781129    2585 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 12:10:55.781667 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.781638    2585 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 12:10:55.781782 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.781488    2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 12:10:55.781782 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.781743    2585 factory.go:55] Registering systemd factory
Sep 09 12:10:55.781782 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.781753    2585 factory.go:223] Registration of the systemd container factory successfully
Sep 09 12:10:55.781934 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.781842    2585 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 12:10:55.781934 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.781849    2585 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 12:10:55.783065 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.783047    2585 factory.go:153] Registering CRI-O factory
Sep 09 12:10:55.783065 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.783067    2585 factory.go:223] Registration of the crio container factory successfully
Sep 09 12:10:55.783201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.783089    2585 factory.go:103] Registering Raw factory
Sep 09 12:10:55.783201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.783101    2585 manager.go:1196] Started watching for new ooms in manager
Sep 09 12:10:55.783666 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.783645    2585 manager.go:349] Starting recovery of all containers
Sep 09 12:10:55.784808 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.784782    2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-207.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 12:10:55.784979 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.784945    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 12:10:55.793201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.793184    2585 manager.go:354] Recovery completed
Sep 09 12:10:55.795643 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.795625    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sscfs"
Sep 09 12:10:55.797086 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.797074    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:10:55.798748 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.798682    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:10:55.798799 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.798751    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:10:55.798799 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.798764    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:10:55.799306 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.799294    2585 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 12:10:55.799306 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.799304    2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 12:10:55.799383 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.799308    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:10:55.799383 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.799318    2585 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:10:55.800434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.800418    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:10:55.800486 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.800454    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:10:55.800486 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.800470    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:10:55.801035 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.800955    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-207.ec2.internal.18d3a5df5fbd079b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-207.ec2.internal,UID:ip-10-0-131-207.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-207.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-207.ec2.internal,},FirstTimestamp:2026-09-09 12:10:55.798732699 +0000 UTC m=+0.440351895,LastTimestamp:2026-09-09 12:10:55.798732699 +0000 UTC m=+0.440351895,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-207.ec2.internal,}"
Sep 09 12:10:55.801503 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.801492    2585 policy_none.go:49] "None policy: Start"
Sep 09 12:10:55.801551 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.801508    2585 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 12:10:55.801551 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.801519    2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 12:10:55.803481 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.803470    2585 policy_none.go:47] "Start"
Sep 09 12:10:55.804390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.804374    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sscfs"
Sep 09 12:10:55.840978 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.840947    2585 manager.go:337] "Starting Device Plugin manager"
Sep 09 12:10:55.866763 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.841005    2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 12:10:55.866763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.841019    2585 server.go:87] "Starting device plugin registration server"
Sep 09 12:10:55.866763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.841304    2585 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 12:10:55.866763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.841318    2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 12:10:55.866763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.841497    2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 12:10:55.866763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.841591    2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 12:10:55.866763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.841601    2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 12:10:55.866763 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.842096    2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 12:10:55.866763 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.842148    2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-207.ec2.internal\" not found"
Sep 09 12:10:55.891379 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.891296    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 12:10:55.892651 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.892632    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 12:10:55.892720 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.892713    2585 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 12:10:55.892767 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.892742    2585 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 12:10:55.892816 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.892783    2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 12:10:55.895134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.895109    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:10:55.942151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.942116    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:10:55.943413 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.943396    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:10:55.943484 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.943427    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:10:55.943484 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.943437    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:10:55.943484 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.943462    2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-207.ec2.internal"
Sep 09 12:10:55.952664 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.952640    2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-207.ec2.internal"
Sep 09 12:10:55.952785 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.952667    2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-207.ec2.internal\": node \"ip-10-0-131-207.ec2.internal\" not found"
Sep 09 12:10:55.983654 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:55.983624    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-207.ec2.internal\" not found"
Sep 09 12:10:55.993310 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.993286    2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal"]
Sep 09 12:10:55.993411 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.993349    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:10:55.994910 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.994893    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:10:55.995001 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.994926    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:10:55.995001 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.994936    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:10:55.997221 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.997208    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:10:55.997343 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.997328    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:55.997388 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.997359    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:10:55.997966 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.997951    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:10:55.998054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.997951    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:10:55.998054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.997994    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:10:55.998054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.998003    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:10:55.998054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.997977    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:10:55.998201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:55.998059    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:10:56.000215 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.000200    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.000310 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.000221    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:10:56.000864 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.000850    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:10:56.000932 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.000878    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:10:56.000932 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.000889    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:10:56.017220 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.017198    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-207.ec2.internal\" not found" node="ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.020554 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.020538    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-207.ec2.internal\" not found" node="ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.083017 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.082987    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba7b0232134df51badacb252b3d6af22-config\") pod \"kube-apiserver-proxy-ip-10-0-131-207.ec2.internal\" (UID: \"ba7b0232134df51badacb252b3d6af22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.083017 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.083020    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b793b7a48d2b0fa6c9c023903323e75e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal\" (UID: \"b793b7a48d2b0fa6c9c023903323e75e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.083161 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.083036    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b793b7a48d2b0fa6c9c023903323e75e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal\" (UID: \"b793b7a48d2b0fa6c9c023903323e75e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.084044 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.084025    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-207.ec2.internal\" not found"
Sep 09 12:10:56.183237 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.183170    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b793b7a48d2b0fa6c9c023903323e75e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal\" (UID: \"b793b7a48d2b0fa6c9c023903323e75e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.183237 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.183204    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba7b0232134df51badacb252b3d6af22-config\") pod \"kube-apiserver-proxy-ip-10-0-131-207.ec2.internal\" (UID: \"ba7b0232134df51badacb252b3d6af22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.183237 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.183223    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b793b7a48d2b0fa6c9c023903323e75e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal\" (UID: \"b793b7a48d2b0fa6c9c023903323e75e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.183415 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.183270    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba7b0232134df51badacb252b3d6af22-config\") pod \"kube-apiserver-proxy-ip-10-0-131-207.ec2.internal\" (UID: \"ba7b0232134df51badacb252b3d6af22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.183415 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.183288    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b793b7a48d2b0fa6c9c023903323e75e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal\" (UID: \"b793b7a48d2b0fa6c9c023903323e75e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.183415 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.183313    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b793b7a48d2b0fa6c9c023903323e75e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal\" (UID: \"b793b7a48d2b0fa6c9c023903323e75e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.184198 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.184184    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-207.ec2.internal\" not found"
Sep 09 12:10:56.284907 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.284870    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-207.ec2.internal\" not found"
Sep 09 12:10:56.320057 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.320029    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.323192 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.323175    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.385122 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.385085    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-207.ec2.internal\" not found"
Sep 09 12:10:56.485650 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.485585    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-207.ec2.internal\" not found"
Sep 09 12:10:56.586083 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.586044    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-207.ec2.internal\" not found"
Sep 09 12:10:56.597324 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.597294    2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:10:56.681361 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.681322    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.697034 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.697004    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:10:56.698727 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.698688    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.706169 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.706140    2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 12:10:56.706299 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.706279    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 12:10:56.706375 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.706350    2585 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a67d490a33aa046a591658b871042c62-b6f9a98096c7d657.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.131.207:46846->52.54.194.22:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal"
Sep 09 12:10:56.755481 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.755457    2585 apiserver.go:52] "Watching apiserver"
Sep 09 12:10:56.763151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.763133    2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 12:10:56.764908 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.764886    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-jwlkw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-j99x9","kube-system/global-pull-secret-syncer-m6ssq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r","openshift-multus/multus-additional-cni-plugins-7wc5n","openshift-multus/multus-hk4nf","openshift-multus/network-metrics-daemon-dj698","openshift-network-diagnostics/network-check-target-n8bz5","openshift-network-operator/iptables-alerter-49nz8","kube-system/konnectivity-agent-vt2db","kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal","openshift-cluster-node-tuning-operator/tuned-99k5f","openshift-dns/node-resolver-7d845"]
Sep 09 12:10:56.767376 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.767359    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:56.769738 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.769693    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 12:10:56.769870 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.769802    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.769870 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.769807    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 12:10:56.770134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.770117    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 12:10:56.770245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.770219    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vddw5\""
Sep 09 12:10:56.771933 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.771913    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 12:10:56.772124 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.772112    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:56.772187 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.772163    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:10:56.772816 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.772782    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 12:10:56.773085 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.773067    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k9gnl\""
Sep 09 12:10:56.773272 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.773255    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 12:10:56.773402 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.773379    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 12:10:56.773482 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.773460    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 12:10:56.775062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.775030    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 12:10:56.775177 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.775163    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.777187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.777155    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.777319 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.777301    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 12:10:56.777749 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.777728    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 12:10:56.777833 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.777799    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 12:10:56.777918 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.777895    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-76cm8\""
Sep 09 12:10:56.779363 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.779344    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.779467 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.779369    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 12:10:56.779467 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.779401    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 12:10:56.779467 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.779404    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 12:10:56.779467 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.779442    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 12:10:56.779743 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.779687    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 12:10:56.779817 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.779794    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7gp54\""
Sep 09 12:10:56.779817 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.779798    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 12:10:56.781608 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.781595    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:10:56.781672 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.781656    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:10:56.781945 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.781927    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g92kb\""
Sep 09 12:10:56.782904 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.782890    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 12:10:56.783767 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.783753    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:10:56.783839 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.783807    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:10:56.785638 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785621    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af64da4a-c9ad-4944-bef3-583521578d0e-host\") pod \"node-ca-jwlkw\" (UID: \"af64da4a-c9ad-4944-bef3-583521578d0e\") " pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:56.785742 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785652    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5a64c743-7066-4939-9f91-88da5ad84d37-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.785742 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785677    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-var-lib-cni-multus\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.785830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785745    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhwmf\" (UniqueName: \"kubernetes.io/projected/5a64c743-7066-4939-9f91-88da5ad84d37-kube-api-access-dhwmf\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.785830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785767    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b727a14-210f-4cfa-afde-83c1813bad01-cni-binary-copy\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.785830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785783    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-multus-socket-dir-parent\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.785830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785804    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-log-socket\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.785830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785826    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/af64da4a-c9ad-4944-bef3-583521578d0e-serviceca\") pod \"node-ca-jwlkw\" (UID: \"af64da4a-c9ad-4944-bef3-583521578d0e\") " pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:56.786040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785846    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a18b39f3-278e-4cc9-97b0-867a633220f3-dbus\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:56.786040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785866    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps8b4\" (UniqueName: \"kubernetes.io/projected/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-kube-api-access-ps8b4\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.786040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785889    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-run-netns\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.786040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785922    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-etc-openvswitch\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.786040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785944    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-ovn-node-metrics-cert\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.786040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785966    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-device-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.786040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.785987    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-etc-selinux\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.786040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786010    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5a64c743-7066-4939-9f91-88da5ad84d37-cni-binary-copy\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.786040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786033    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5a64c743-7066-4939-9f91-88da5ad84d37-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.786434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786057    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-kubelet\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.786434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786101    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-run-systemd\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.786434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786133    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-env-overrides\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.786434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786156    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:56.786434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786156    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-sys-fs\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.786434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786427    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-cnibin\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786448    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-var-lib-kubelet\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786471    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c4pr\" (UniqueName: \"kubernetes.io/projected/af64da4a-c9ad-4944-bef3-583521578d0e-kube-api-access-4c4pr\") pod \"node-ca-jwlkw\" (UID: \"af64da4a-c9ad-4944-bef3-583521578d0e\") " pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786495    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-run-openvswitch\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786515    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-cni-netd\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786555    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786615    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-node-log\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786640    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-multus-cni-dir\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786663    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5a64c743-7066-4939-9f91-88da5ad84d37-cnibin\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786716    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3b727a14-210f-4cfa-afde-83c1813bad01-multus-daemon-config\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786746    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-ovnkube-config\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786800    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5a64c743-7066-4939-9f91-88da5ad84d37-system-cni-dir\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786834    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-run-ovn\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786848    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-cni-bin\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786864    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-var-lib-cni-bin\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786898    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:10:56.787201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786935    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-systemd-units\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786966    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-ovnkube-script-lib\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.786984    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787012    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787040    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvcb8\" (UniqueName: \"kubernetes.io/projected/13374047-806d-4ebf-966f-56beae8c6ba5-kube-api-access-kvcb8\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787064    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-system-cni-dir\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787077    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-slash\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787089    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-hostroot\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787104    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5a64c743-7066-4939-9f91-88da5ad84d37-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787125    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-run-netns\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787146    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-etc-kubernetes\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787171    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4wdj\" (UniqueName: \"kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj\") pod \"network-check-target-n8bz5\" (UID: \"cee4188b-7e72-4b33-bde9-d1b2a585d0b5\") " pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787184    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-socket-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787198    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-multus-conf-dir\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787220    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwvqc\" (UniqueName: \"kubernetes.io/projected/0238389f-6661-453f-ba7b-b33a3c34271c-kube-api-access-mwvqc\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787252    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a18b39f3-278e-4cc9-97b0-867a633220f3-kubelet-config\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:56.787858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787267    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-var-lib-openvswitch\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.788369 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787286    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-run-multus-certs\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.788369 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787305    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n582m\" (UniqueName: \"kubernetes.io/projected/3b727a14-210f-4cfa-afde-83c1813bad01-kube-api-access-n582m\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.788369 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787318    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-run-ovn-kubernetes\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.788369 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787364    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-registration-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.788369 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.787384    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-run-k8s-cni-cncf-io\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.788590 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.788570    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:10:56.788666 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.788589    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cndk5\""
Sep 09 12:10:56.788666 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.788621    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 12:10:56.788666 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.788636    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:10:56.788666 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.788571    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 12:10:56.790995 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.790975    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.791080 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.791005    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9lb5h\""
Sep 09 12:10:56.791554 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.791538    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 12:10:56.791631 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.791574    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 12:10:56.793213 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.793195    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 12:10:56.793288 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.793230    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:10:56.793288 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.793243    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jhk46\""
Sep 09 12:10:56.793389 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.793349    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:56.796805 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.796791    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 12:10:56.796884 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.796821    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d7w86\""
Sep 09 12:10:56.796963 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.796951    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 12:10:56.800192 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.800177    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:10:56.807178 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.807135    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 12:05:55 +0000 UTC" deadline="2028-02-13 12:28:32.540906791 +0000 UTC"
Sep 09 12:10:56.807249 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.807177    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12528h17m35.733733321s"
Sep 09 12:10:56.826910 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.826892    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w8g5c"
Sep 09 12:10:56.836958 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.836936    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w8g5c"
Sep 09 12:10:56.882480 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.882450    2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 12:10:56.888245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888219    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-var-lib-openvswitch\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.888355 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888258    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-run-multus-certs\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.888355 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888286    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n582m\" (UniqueName: \"kubernetes.io/projected/3b727a14-210f-4cfa-afde-83c1813bad01-kube-api-access-n582m\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.888355 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888313    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-sysctl-conf\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.888355 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888335    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-systemd\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.888355 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888344    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-var-lib-openvswitch\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.888572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888357    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-run-ovn-kubernetes\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.888572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888362    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-run-multus-certs\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.888572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888397    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-registration-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.888572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888437    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-run-ovn-kubernetes\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.888572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888474    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-run-k8s-cni-cncf-io\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.888572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888497    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp5z8\" (UniqueName: \"kubernetes.io/projected/72f2bf19-006c-41ff-9f3e-e41c1d113274-kube-api-access-dp5z8\") pod \"node-resolver-7d845\" (UID: \"72f2bf19-006c-41ff-9f3e-e41c1d113274\") " pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:56.888572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888521    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af64da4a-c9ad-4944-bef3-583521578d0e-host\") pod \"node-ca-jwlkw\" (UID: \"af64da4a-c9ad-4944-bef3-583521578d0e\") " pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:56.888572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888540    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-registration-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.888572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888562    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5a64c743-7066-4939-9f91-88da5ad84d37-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.888572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888569    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af64da4a-c9ad-4944-bef3-583521578d0e-host\") pod \"node-ca-jwlkw\" (UID: \"af64da4a-c9ad-4944-bef3-583521578d0e\") " pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888581    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-run-k8s-cni-cncf-io\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888612    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-var-lib-cni-multus\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888654    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-host\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888666    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-var-lib-cni-multus\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888722    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72f2bf19-006c-41ff-9f3e-e41c1d113274-hosts-file\") pod \"node-resolver-7d845\" (UID: \"72f2bf19-006c-41ff-9f3e-e41c1d113274\") " pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888748    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f2bf19-006c-41ff-9f3e-e41c1d113274-tmp-dir\") pod \"node-resolver-7d845\" (UID: \"72f2bf19-006c-41ff-9f3e-e41c1d113274\") " pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888766    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhwmf\" (UniqueName: \"kubernetes.io/projected/5a64c743-7066-4939-9f91-88da5ad84d37-kube-api-access-dhwmf\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888789    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b727a14-210f-4cfa-afde-83c1813bad01-cni-binary-copy\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888815    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-multus-socket-dir-parent\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888838    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-log-socket\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888862    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/af64da4a-c9ad-4944-bef3-583521578d0e-serviceca\") pod \"node-ca-jwlkw\" (UID: \"af64da4a-c9ad-4944-bef3-583521578d0e\") " pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888885    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a18b39f3-278e-4cc9-97b0-867a633220f3-dbus\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888904    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-multus-socket-dir-parent\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888910    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps8b4\" (UniqueName: \"kubernetes.io/projected/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-kube-api-access-ps8b4\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888935    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/243df091-5947-47d0-bd58-fd2dd2bfe015-iptables-alerter-script\") pod \"iptables-alerter-49nz8\" (UID: \"243df091-5947-47d0-bd58-fd2dd2bfe015\") " pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888959    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-run-netns\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.889054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.888981    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-etc-openvswitch\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889005    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-ovn-node-metrics-cert\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889027    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-device-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889049    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-etc-selinux\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889081    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5a64c743-7066-4939-9f91-88da5ad84d37-cni-binary-copy\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889106    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5a64c743-7066-4939-9f91-88da5ad84d37-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889130    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ce58793-93d8-4b04-af82-66521edd67dc-agent-certs\") pod \"konnectivity-agent-vt2db\" (UID: \"9ce58793-93d8-4b04-af82-66521edd67dc\") " pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889156    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a18b39f3-278e-4cc9-97b0-867a633220f3-dbus\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889162    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-kubelet\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889187    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-run-systemd\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889204    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-env-overrides\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889204    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-kubelet\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889219    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-sys-fs\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889234    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-cnibin\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889249    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-var-lib-kubelet\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889257    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-run-netns\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889264    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-run\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.889654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889279    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6499166f-4e4d-4db6-a1c7-5ef8a9502054-tmp\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889295    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c4pr\" (UniqueName: \"kubernetes.io/projected/af64da4a-c9ad-4944-bef3-583521578d0e-kube-api-access-4c4pr\") pod \"node-ca-jwlkw\" (UID: \"af64da4a-c9ad-4944-bef3-583521578d0e\") " pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889298    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-etc-openvswitch\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889310    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-run-openvswitch\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889335    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-run-openvswitch\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889359    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/af64da4a-c9ad-4944-bef3-583521578d0e-serviceca\") pod \"node-ca-jwlkw\" (UID: \"af64da4a-c9ad-4944-bef3-583521578d0e\") " pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889372    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b727a14-210f-4cfa-afde-83c1813bad01-cni-binary-copy\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889444    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-log-socket\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889481    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-var-lib-kubelet\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889526    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-sys-fs\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889567    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-run-systemd\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889603    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-cnibin\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889654    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-env-overrides\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889656    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-device-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889689    2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889693    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-etc-selinux\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889762    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-cni-netd\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.889836    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-cni-netd\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.890456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890098    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-modprobe-d\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890166    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-lib-modules\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890198    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890240    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-node-log\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890266    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-multus-cni-dir\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.890285    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890291    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qrwz\" (UniqueName: \"kubernetes.io/projected/6499166f-4e4d-4db6-a1c7-5ef8a9502054-kube-api-access-4qrwz\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890335    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5a64c743-7066-4939-9f91-88da5ad84d37-cnibin\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.890352    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret podName:a18b39f3-278e-4cc9-97b0-867a633220f3 nodeName:}" failed. No retries permitted until 2026-09-09 12:10:57.390335296 +0000 UTC m=+2.031954496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret") pod "global-pull-secret-syncer-m6ssq" (UID: "a18b39f3-278e-4cc9-97b0-867a633220f3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890361    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5a64c743-7066-4939-9f91-88da5ad84d37-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890377    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3b727a14-210f-4cfa-afde-83c1813bad01-multus-daemon-config\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890380    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5a64c743-7066-4939-9f91-88da5ad84d37-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890396    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5a64c743-7066-4939-9f91-88da5ad84d37-cnibin\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890403    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5lrx\" (UniqueName: \"kubernetes.io/projected/243df091-5947-47d0-bd58-fd2dd2bfe015-kube-api-access-b5lrx\") pod \"iptables-alerter-49nz8\" (UID: \"243df091-5947-47d0-bd58-fd2dd2bfe015\") " pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890406    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5a64c743-7066-4939-9f91-88da5ad84d37-cni-binary-copy\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890424    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-ovnkube-config\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.891373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890445    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-node-log\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890456    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5a64c743-7066-4939-9f91-88da5ad84d37-system-cni-dir\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890492    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5a64c743-7066-4939-9f91-88da5ad84d37-system-cni-dir\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890495    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/243df091-5947-47d0-bd58-fd2dd2bfe015-host-slash\") pod \"iptables-alerter-49nz8\" (UID: \"243df091-5947-47d0-bd58-fd2dd2bfe015\") " pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890558    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-run-ovn\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890564    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-multus-cni-dir\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890592    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-cni-bin\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890599    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-run-ovn\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890619    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-var-lib-cni-bin\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890644    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890672    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-systemd-units\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890729    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-ovnkube-script-lib\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890755    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890783    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890794    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-cni-bin\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890807    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvcb8\" (UniqueName: \"kubernetes.io/projected/13374047-806d-4ebf-966f-56beae8c6ba5-kube-api-access-kvcb8\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890757    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-var-lib-cni-bin\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.891939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890828    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3b727a14-210f-4cfa-afde-83c1813bad01-multus-daemon-config\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890851    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890861    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-ovnkube-config\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890831    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-system-cni-dir\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890906    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-kubernetes\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890926    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-sysctl-d\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.890940    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890879    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.890992    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs podName:0238389f-6661-453f-ba7b-b33a3c34271c nodeName:}" failed. No retries permitted until 2026-09-09 12:10:57.390977653 +0000 UTC m=+2.032596836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs") pod "network-metrics-daemon-dj698" (UID: "0238389f-6661-453f-ba7b-b33a3c34271c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890990    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-systemd-units\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.890943    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-tuned\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891030    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-slash\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891050    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-hostroot\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891053    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-system-cni-dir\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891066    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-sysconfig\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891097    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-host-slash\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891085    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5a64c743-7066-4939-9f91-88da5ad84d37-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.892412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891133    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-hostroot\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891152    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-run-netns\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891177    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-etc-kubernetes\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891226    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-host-run-netns\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891200    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ce58793-93d8-4b04-af82-66521edd67dc-konnectivity-ca\") pod \"konnectivity-agent-vt2db\" (UID: \"9ce58793-93d8-4b04-af82-66521edd67dc\") " pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891279    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5a64c743-7066-4939-9f91-88da5ad84d37-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891283    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-sys\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891314    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-etc-kubernetes\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891328    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-ovnkube-script-lib\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891351    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wdj\" (UniqueName: \"kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj\") pod \"network-check-target-n8bz5\" (UID: \"cee4188b-7e72-4b33-bde9-d1b2a585d0b5\") " pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891376    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-socket-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891395    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-multus-conf-dir\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891412    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-var-lib-kubelet\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891436    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwvqc\" (UniqueName: \"kubernetes.io/projected/0238389f-6661-453f-ba7b-b33a3c34271c-kube-api-access-mwvqc\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891458    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a18b39f3-278e-4cc9-97b0-867a633220f3-kubelet-config\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891467    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b727a14-210f-4cfa-afde-83c1813bad01-multus-conf-dir\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891512    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13374047-806d-4ebf-966f-56beae8c6ba5-socket-dir\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.892899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.891552    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a18b39f3-278e-4cc9-97b0-867a633220f3-kubelet-config\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:56.893359 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.893283    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-ovn-node-metrics-cert\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.900992 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.900951    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:10:56.900992 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.900994    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:10:56.901157 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.901008    2585 projected.go:196] Error preparing data for projected volume kube-api-access-b4wdj for pod openshift-network-diagnostics/network-check-target-n8bz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:10:56.901157 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:56.901148    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj podName:cee4188b-7e72-4b33-bde9-d1b2a585d0b5 nodeName:}" failed. No retries permitted until 2026-09-09 12:10:57.401099477 +0000 UTC m=+2.042718678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b4wdj" (UniqueName: "kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj") pod "network-check-target-n8bz5" (UID: "cee4188b-7e72-4b33-bde9-d1b2a585d0b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:10:56.903229 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.902809    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhwmf\" (UniqueName: \"kubernetes.io/projected/5a64c743-7066-4939-9f91-88da5ad84d37-kube-api-access-dhwmf\") pod \"multus-additional-cni-plugins-7wc5n\" (UID: \"5a64c743-7066-4939-9f91-88da5ad84d37\") " pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:56.903229 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.903146    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps8b4\" (UniqueName: \"kubernetes.io/projected/fc9f3a1d-ab68-49a3-b267-5cc0113fcc21-kube-api-access-ps8b4\") pod \"ovnkube-node-j99x9\" (UID: \"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21\") " pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:56.903548 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.903529    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n582m\" (UniqueName: \"kubernetes.io/projected/3b727a14-210f-4cfa-afde-83c1813bad01-kube-api-access-n582m\") pod \"multus-hk4nf\" (UID: \"3b727a14-210f-4cfa-afde-83c1813bad01\") " pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:56.903662 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.903644    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvcb8\" (UniqueName: \"kubernetes.io/projected/13374047-806d-4ebf-966f-56beae8c6ba5-kube-api-access-kvcb8\") pod \"aws-ebs-csi-driver-node-qvj9r\" (UID: \"13374047-806d-4ebf-966f-56beae8c6ba5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:56.903722 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.903641    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c4pr\" (UniqueName: \"kubernetes.io/projected/af64da4a-c9ad-4944-bef3-583521578d0e-kube-api-access-4c4pr\") pod \"node-ca-jwlkw\" (UID: \"af64da4a-c9ad-4944-bef3-583521578d0e\") " pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:56.904470 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.904455    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwvqc\" (UniqueName: \"kubernetes.io/projected/0238389f-6661-453f-ba7b-b33a3c34271c-kube-api-access-mwvqc\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:10:56.991945 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.991908    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/243df091-5947-47d0-bd58-fd2dd2bfe015-iptables-alerter-script\") pod \"iptables-alerter-49nz8\" (UID: \"243df091-5947-47d0-bd58-fd2dd2bfe015\") " pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:56.992204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.991954    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ce58793-93d8-4b04-af82-66521edd67dc-agent-certs\") pod \"konnectivity-agent-vt2db\" (UID: \"9ce58793-93d8-4b04-af82-66521edd67dc\") " pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:10:56.992204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992069    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-run\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992109    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6499166f-4e4d-4db6-a1c7-5ef8a9502054-tmp\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992138    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-modprobe-d\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992162    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-lib-modules\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992182    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-run\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992202    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qrwz\" (UniqueName: \"kubernetes.io/projected/6499166f-4e4d-4db6-a1c7-5ef8a9502054-kube-api-access-4qrwz\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992228    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5lrx\" (UniqueName: \"kubernetes.io/projected/243df091-5947-47d0-bd58-fd2dd2bfe015-kube-api-access-b5lrx\") pod \"iptables-alerter-49nz8\" (UID: \"243df091-5947-47d0-bd58-fd2dd2bfe015\") " pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992254    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/243df091-5947-47d0-bd58-fd2dd2bfe015-host-slash\") pod \"iptables-alerter-49nz8\" (UID: \"243df091-5947-47d0-bd58-fd2dd2bfe015\") " pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992298    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-kubernetes\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992313    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-modprobe-d\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992320    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-sysctl-d\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992345    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-tuned\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992369    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-sysconfig\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992394    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ce58793-93d8-4b04-af82-66521edd67dc-konnectivity-ca\") pod \"konnectivity-agent-vt2db\" (UID: \"9ce58793-93d8-4b04-af82-66521edd67dc\") " pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992417    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-sys\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992451    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-var-lib-kubelet\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992481    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-sysctl-conf\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992504    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-systemd\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992530    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp5z8\" (UniqueName: \"kubernetes.io/projected/72f2bf19-006c-41ff-9f3e-e41c1d113274-kube-api-access-dp5z8\") pod \"node-resolver-7d845\" (UID: \"72f2bf19-006c-41ff-9f3e-e41c1d113274\") " pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992555    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-host\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.992574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992579    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72f2bf19-006c-41ff-9f3e-e41c1d113274-hosts-file\") pod \"node-resolver-7d845\" (UID: \"72f2bf19-006c-41ff-9f3e-e41c1d113274\") " pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:56.993225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992602    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f2bf19-006c-41ff-9f3e-e41c1d113274-tmp-dir\") pod \"node-resolver-7d845\" (UID: \"72f2bf19-006c-41ff-9f3e-e41c1d113274\") " pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:56.993225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992603    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/243df091-5947-47d0-bd58-fd2dd2bfe015-iptables-alerter-script\") pod \"iptables-alerter-49nz8\" (UID: \"243df091-5947-47d0-bd58-fd2dd2bfe015\") " pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:56.993225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992870    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-sysctl-d\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.993225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992313    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-lib-modules\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.993225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992932    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/243df091-5947-47d0-bd58-fd2dd2bfe015-host-slash\") pod \"iptables-alerter-49nz8\" (UID: \"243df091-5947-47d0-bd58-fd2dd2bfe015\") " pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:56.993225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.992985    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-kubernetes\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.993225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.993055    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-sysctl-conf\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.993225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.993125    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-sys\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.993225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.993177    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-var-lib-kubelet\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.993719 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.993233    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-systemd\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.993719 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.993236    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-sysconfig\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.993719 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.993283    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6499166f-4e4d-4db6-a1c7-5ef8a9502054-host\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.993719 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.993334    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72f2bf19-006c-41ff-9f3e-e41c1d113274-hosts-file\") pod \"node-resolver-7d845\" (UID: \"72f2bf19-006c-41ff-9f3e-e41c1d113274\") " pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:56.993934 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.993744    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ce58793-93d8-4b04-af82-66521edd67dc-konnectivity-ca\") pod \"konnectivity-agent-vt2db\" (UID: \"9ce58793-93d8-4b04-af82-66521edd67dc\") " pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:10:56.994124 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.994096    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f2bf19-006c-41ff-9f3e-e41c1d113274-tmp-dir\") pod \"node-resolver-7d845\" (UID: \"72f2bf19-006c-41ff-9f3e-e41c1d113274\") " pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:56.995225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.995201    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ce58793-93d8-4b04-af82-66521edd67dc-agent-certs\") pod \"konnectivity-agent-vt2db\" (UID: \"9ce58793-93d8-4b04-af82-66521edd67dc\") " pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:10:56.995352 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.995301    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6499166f-4e4d-4db6-a1c7-5ef8a9502054-etc-tuned\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:56.995416 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:56.995350    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6499166f-4e4d-4db6-a1c7-5ef8a9502054-tmp\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:57.002640 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.002615    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp5z8\" (UniqueName: \"kubernetes.io/projected/72f2bf19-006c-41ff-9f3e-e41c1d113274-kube-api-access-dp5z8\") pod \"node-resolver-7d845\" (UID: \"72f2bf19-006c-41ff-9f3e-e41c1d113274\") " pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:57.002640 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.002643    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5lrx\" (UniqueName: \"kubernetes.io/projected/243df091-5947-47d0-bd58-fd2dd2bfe015-kube-api-access-b5lrx\") pod \"iptables-alerter-49nz8\" (UID: \"243df091-5947-47d0-bd58-fd2dd2bfe015\") " pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:57.002640 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.002623    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qrwz\" (UniqueName: \"kubernetes.io/projected/6499166f-4e4d-4db6-a1c7-5ef8a9502054-kube-api-access-4qrwz\") pod \"tuned-99k5f\" (UID: \"6499166f-4e4d-4db6-a1c7-5ef8a9502054\") " pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:57.039055 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.038868    2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:10:57.090295 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.090261    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jwlkw"
Sep 09 12:10:57.106124 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.106100    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:10:57.111366 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.111329    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb793b7a48d2b0fa6c9c023903323e75e.slice/crio-3d31c3dff3ff13d520a459d29ae10f3f323ec2a087556c32fd29cc45160eb6e0 WatchSource:0}: Error finding container 3d31c3dff3ff13d520a459d29ae10f3f323ec2a087556c32fd29cc45160eb6e0: Status 404 returned error can't find the container with id 3d31c3dff3ff13d520a459d29ae10f3f323ec2a087556c32fd29cc45160eb6e0
Sep 09 12:10:57.112548 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.112515    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba7b0232134df51badacb252b3d6af22.slice/crio-1433ce6ebeceb3ae1ba8aa08868a07e77c8d8802a28420257af74a4411bf9739 WatchSource:0}: Error finding container 1433ce6ebeceb3ae1ba8aa08868a07e77c8d8802a28420257af74a4411bf9739: Status 404 returned error can't find the container with id 1433ce6ebeceb3ae1ba8aa08868a07e77c8d8802a28420257af74a4411bf9739
Sep 09 12:10:57.113232 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.113212    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf64da4a_c9ad_4944_bef3_583521578d0e.slice/crio-a0805528fb4040ce276868f7f38411ba330f4b7b4debb13c28bde5298e36e1a0 WatchSource:0}: Error finding container a0805528fb4040ce276868f7f38411ba330f4b7b4debb13c28bde5298e36e1a0: Status 404 returned error can't find the container with id a0805528fb4040ce276868f7f38411ba330f4b7b4debb13c28bde5298e36e1a0
Sep 09 12:10:57.117993 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.117975    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:10:57.118470 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.118448    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9f3a1d_ab68_49a3_b267_5cc0113fcc21.slice/crio-8691401a6da1daa210f3ca05fec1c7ef9b9f82c88b2fc37eea3eb562729ccb7c WatchSource:0}: Error finding container 8691401a6da1daa210f3ca05fec1c7ef9b9f82c88b2fc37eea3eb562729ccb7c: Status 404 returned error can't find the container with id 8691401a6da1daa210f3ca05fec1c7ef9b9f82c88b2fc37eea3eb562729ccb7c
Sep 09 12:10:57.125882 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.125862    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r"
Sep 09 12:10:57.135734 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.135709    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13374047_806d_4ebf_966f_56beae8c6ba5.slice/crio-b006708d2b47c06141c31490c9761a8a892081d52fcf783f69b9ca9c6d77e4a9 WatchSource:0}: Error finding container b006708d2b47c06141c31490c9761a8a892081d52fcf783f69b9ca9c6d77e4a9: Status 404 returned error can't find the container with id b006708d2b47c06141c31490c9761a8a892081d52fcf783f69b9ca9c6d77e4a9
Sep 09 12:10:57.139132 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.139115    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7wc5n"
Sep 09 12:10:57.145003 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.144988    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hk4nf"
Sep 09 12:10:57.147123 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.147100    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a64c743_7066_4939_9f91_88da5ad84d37.slice/crio-95c15fa677e78e92982ebdca0c7f83d08ef573471ef54e3f1b96f23f673bfee5 WatchSource:0}: Error finding container 95c15fa677e78e92982ebdca0c7f83d08ef573471ef54e3f1b96f23f673bfee5: Status 404 returned error can't find the container with id 95c15fa677e78e92982ebdca0c7f83d08ef573471ef54e3f1b96f23f673bfee5
Sep 09 12:10:57.149005 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.148983    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-49nz8"
Sep 09 12:10:57.153571 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.153548    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b727a14_210f_4cfa_afde_83c1813bad01.slice/crio-e352f1084f82ecc9a6ce0abe0f5103008aedb8239a308c4f337c6b6b5ecccea9 WatchSource:0}: Error finding container e352f1084f82ecc9a6ce0abe0f5103008aedb8239a308c4f337c6b6b5ecccea9: Status 404 returned error can't find the container with id e352f1084f82ecc9a6ce0abe0f5103008aedb8239a308c4f337c6b6b5ecccea9
Sep 09 12:10:57.158324 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.158305    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243df091_5947_47d0_bd58_fd2dd2bfe015.slice/crio-9987af292ef372f1df2650d0f12360fe0935a3dc646b604f52583a4a9c78e03f WatchSource:0}: Error finding container 9987af292ef372f1df2650d0f12360fe0935a3dc646b604f52583a4a9c78e03f: Status 404 returned error can't find the container with id 9987af292ef372f1df2650d0f12360fe0935a3dc646b604f52583a4a9c78e03f
Sep 09 12:10:57.190686 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.190649    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:10:57.200021 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.200002    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-99k5f"
Sep 09 12:10:57.200521 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.200500    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce58793_93d8_4b04_af82_66521edd67dc.slice/crio-248fe7de15bda8a7c2897e7a74f1e84df62bd61e34e5259355f23271d063a7aa WatchSource:0}: Error finding container 248fe7de15bda8a7c2897e7a74f1e84df62bd61e34e5259355f23271d063a7aa: Status 404 returned error can't find the container with id 248fe7de15bda8a7c2897e7a74f1e84df62bd61e34e5259355f23271d063a7aa
Sep 09 12:10:57.210367 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.210343    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6499166f_4e4d_4db6_a1c7_5ef8a9502054.slice/crio-51f0d39867de24269643d0be27a6ee1c8612dbcb498fc60827e5272879378d38 WatchSource:0}: Error finding container 51f0d39867de24269643d0be27a6ee1c8612dbcb498fc60827e5272879378d38: Status 404 returned error can't find the container with id 51f0d39867de24269643d0be27a6ee1c8612dbcb498fc60827e5272879378d38
Sep 09 12:10:57.211295 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.211274    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7d845"
Sep 09 12:10:57.219638 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:10:57.219617    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f2bf19_006c_41ff_9f3e_e41c1d113274.slice/crio-8dec6346392938f46f267faf7b2b15d0b6301e85627b7d0cf52979e8a245975c WatchSource:0}: Error finding container 8dec6346392938f46f267faf7b2b15d0b6301e85627b7d0cf52979e8a245975c: Status 404 returned error can't find the container with id 8dec6346392938f46f267faf7b2b15d0b6301e85627b7d0cf52979e8a245975c
Sep 09 12:10:57.221207 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.221190    2585 kuberuntime_manager.go:1459] "Unhandled Error" err="container dns-node-resolver start failed in pod node-resolver-7d845_openshift-dns(72f2bf19-006c-41ff-9f3e-e41c1d113274): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 09 12:10:57.221262 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.221221    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-dns/node-resolver-7d845" podUID="72f2bf19-006c-41ff-9f3e-e41c1d113274"
Sep 09 12:10:57.271968 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.271929    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:10:57.395143 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.395040    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:57.395143 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.395088    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:10:57.395346 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.395202    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:10:57.395346 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.395203    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:10:57.395346 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.395270    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs podName:0238389f-6661-453f-ba7b-b33a3c34271c nodeName:}" failed. No retries permitted until 2026-09-09 12:10:58.395251239 +0000 UTC m=+3.036870426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs") pod "network-metrics-daemon-dj698" (UID: "0238389f-6661-453f-ba7b-b33a3c34271c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:10:57.395346 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.395284    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret podName:a18b39f3-278e-4cc9-97b0-867a633220f3 nodeName:}" failed. No retries permitted until 2026-09-09 12:10:58.395277616 +0000 UTC m=+3.036896799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret") pod "global-pull-secret-syncer-m6ssq" (UID: "a18b39f3-278e-4cc9-97b0-867a633220f3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:10:57.495929 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.495892    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wdj\" (UniqueName: \"kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj\") pod \"network-check-target-n8bz5\" (UID: \"cee4188b-7e72-4b33-bde9-d1b2a585d0b5\") " pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:10:57.496087 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.496070    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:10:57.496134 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.496093    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:10:57.496134 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.496103    2585 projected.go:196] Error preparing data for projected volume kube-api-access-b4wdj for pod openshift-network-diagnostics/network-check-target-n8bz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:10:57.496206 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.496158    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj podName:cee4188b-7e72-4b33-bde9-d1b2a585d0b5 nodeName:}" failed. No retries permitted until 2026-09-09 12:10:58.496142307 +0000 UTC m=+3.137761495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4wdj" (UniqueName: "kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj") pod "network-check-target-n8bz5" (UID: "cee4188b-7e72-4b33-bde9-d1b2a585d0b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:10:57.840289 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.840218    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 12:05:56 +0000 UTC" deadline="2028-02-13 08:23:53.08239631 +0000 UTC"
Sep 09 12:10:57.840289 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.840287    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12524h12m55.242113672s"
Sep 09 12:10:57.894072 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.894037    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:10:57.894253 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.894169    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:10:57.901390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.901331    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hk4nf" event={"ID":"3b727a14-210f-4cfa-afde-83c1813bad01","Type":"ContainerStarted","Data":"e352f1084f82ecc9a6ce0abe0f5103008aedb8239a308c4f337c6b6b5ecccea9"}
Sep 09 12:10:57.902970 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.902947    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wc5n" event={"ID":"5a64c743-7066-4939-9f91-88da5ad84d37","Type":"ContainerStarted","Data":"95c15fa677e78e92982ebdca0c7f83d08ef573471ef54e3f1b96f23f673bfee5"}
Sep 09 12:10:57.904640 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.904586    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r" event={"ID":"13374047-806d-4ebf-966f-56beae8c6ba5","Type":"ContainerStarted","Data":"b006708d2b47c06141c31490c9761a8a892081d52fcf783f69b9ca9c6d77e4a9"}
Sep 09 12:10:57.906152 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.906127    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7d845" event={"ID":"72f2bf19-006c-41ff-9f3e-e41c1d113274","Type":"ContainerStarted","Data":"8dec6346392938f46f267faf7b2b15d0b6301e85627b7d0cf52979e8a245975c"}
Sep 09 12:10:57.909047 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.908988    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-49nz8" event={"ID":"243df091-5947-47d0-bd58-fd2dd2bfe015","Type":"ContainerStarted","Data":"9987af292ef372f1df2650d0f12360fe0935a3dc646b604f52583a4a9c78e03f"}
Sep 09 12:10:57.910122 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:57.910029    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:40bc1b9c6b608369978e9f165c5e22d7a75fd91b3480cba60cd275228d721616\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-dns/node-resolver-7d845" podUID="72f2bf19-006c-41ff-9f3e-e41c1d113274"
Sep 09 12:10:57.920818 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.915067    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9" event={"ID":"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21","Type":"ContainerStarted","Data":"8691401a6da1daa210f3ca05fec1c7ef9b9f82c88b2fc37eea3eb562729ccb7c"}
Sep 09 12:10:57.920818 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.917697    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal" event={"ID":"b793b7a48d2b0fa6c9c023903323e75e","Type":"ContainerStarted","Data":"3d31c3dff3ff13d520a459d29ae10f3f323ec2a087556c32fd29cc45160eb6e0"}
Sep 09 12:10:57.921321 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.921225    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jwlkw" event={"ID":"af64da4a-c9ad-4944-bef3-583521578d0e","Type":"ContainerStarted","Data":"a0805528fb4040ce276868f7f38411ba330f4b7b4debb13c28bde5298e36e1a0"}
Sep 09 12:10:57.925111 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.925087    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal" event={"ID":"ba7b0232134df51badacb252b3d6af22","Type":"ContainerStarted","Data":"1433ce6ebeceb3ae1ba8aa08868a07e77c8d8802a28420257af74a4411bf9739"}
Sep 09 12:10:57.927696 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.927668    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-99k5f" event={"ID":"6499166f-4e4d-4db6-a1c7-5ef8a9502054","Type":"ContainerStarted","Data":"51f0d39867de24269643d0be27a6ee1c8612dbcb498fc60827e5272879378d38"}
Sep 09 12:10:57.929599 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:57.929573    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vt2db" event={"ID":"9ce58793-93d8-4b04-af82-66521edd67dc","Type":"ContainerStarted","Data":"248fe7de15bda8a7c2897e7a74f1e84df62bd61e34e5259355f23271d063a7aa"}
Sep 09 12:10:58.146031 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:58.145933    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:10:58.404697 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:58.403938    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:58.404697 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:58.404002    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:10:58.404697 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:58.404122    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:10:58.404697 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:58.404149    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:10:58.404697 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:58.404222    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs podName:0238389f-6661-453f-ba7b-b33a3c34271c nodeName:}" failed. No retries permitted until 2026-09-09 12:11:00.404201645 +0000 UTC m=+5.045820832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs") pod "network-metrics-daemon-dj698" (UID: "0238389f-6661-453f-ba7b-b33a3c34271c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:10:58.404697 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:58.404242    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret podName:a18b39f3-278e-4cc9-97b0-867a633220f3 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:00.404232219 +0000 UTC m=+5.045851450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret") pod "global-pull-secret-syncer-m6ssq" (UID: "a18b39f3-278e-4cc9-97b0-867a633220f3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:10:58.505148 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:58.505110    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wdj\" (UniqueName: \"kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj\") pod \"network-check-target-n8bz5\" (UID: \"cee4188b-7e72-4b33-bde9-d1b2a585d0b5\") " pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:10:58.505348 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:58.505296    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:10:58.505348 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:58.505317    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:10:58.505348 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:58.505331    2585 projected.go:196] Error preparing data for projected volume kube-api-access-b4wdj for pod openshift-network-diagnostics/network-check-target-n8bz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:10:58.505526 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:58.505396    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj podName:cee4188b-7e72-4b33-bde9-d1b2a585d0b5 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:00.505375302 +0000 UTC m=+5.146994503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4wdj" (UniqueName: "kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj") pod "network-check-target-n8bz5" (UID: "cee4188b-7e72-4b33-bde9-d1b2a585d0b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:10:58.840938 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:58.840846    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 12:05:56 +0000 UTC" deadline="2028-05-28 16:00:16.2355999 +0000 UTC"
Sep 09 12:10:58.840938 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:58.840904    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15051h49m17.394700285s"
Sep 09 12:10:58.893802 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:58.893739    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:10:58.893955 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:58.893853    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:10:58.894251 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:58.894229    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:10:58.894372 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:58.894325    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:10:59.895425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:10:59.895392    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:10:59.895982 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:10:59.895538    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:00.423053 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:00.423015    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:00.423254 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:00.423071    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:00.423254 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:00.423202    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:11:00.423366 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:00.423261    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs podName:0238389f-6661-453f-ba7b-b33a3c34271c nodeName:}" failed. No retries permitted until 2026-09-09 12:11:04.423242235 +0000 UTC m=+9.064861417 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs") pod "network-metrics-daemon-dj698" (UID: "0238389f-6661-453f-ba7b-b33a3c34271c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:11:00.423661 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:00.423642    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:11:00.423783 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:00.423713    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret podName:a18b39f3-278e-4cc9-97b0-867a633220f3 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:04.423683929 +0000 UTC m=+9.065303112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret") pod "global-pull-secret-syncer-m6ssq" (UID: "a18b39f3-278e-4cc9-97b0-867a633220f3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:11:00.524264 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:00.524228    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wdj\" (UniqueName: \"kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj\") pod \"network-check-target-n8bz5\" (UID: \"cee4188b-7e72-4b33-bde9-d1b2a585d0b5\") " pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:00.524416 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:00.524400    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:11:00.524456 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:00.524419    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:11:00.524456 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:00.524431    2585 projected.go:196] Error preparing data for projected volume kube-api-access-b4wdj for pod openshift-network-diagnostics/network-check-target-n8bz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:11:00.524518 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:00.524489    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj podName:cee4188b-7e72-4b33-bde9-d1b2a585d0b5 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:04.524470693 +0000 UTC m=+9.166089897 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4wdj" (UniqueName: "kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj") pod "network-check-target-n8bz5" (UID: "cee4188b-7e72-4b33-bde9-d1b2a585d0b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:11:00.893887 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:00.893488    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:00.893887 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:00.893589    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:00.893887 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:00.893808    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:00.894131 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:00.893952    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:01.894114 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:01.893693    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:01.894114 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:01.893842    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:02.894467 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:02.893836    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:02.894467 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:02.893975    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:02.894467 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:02.894360    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:02.894467 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:02.894429    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:03.894914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:03.894883    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:03.895330 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:03.895008    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:04.462270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:04.462188    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:04.462270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:04.462247    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:04.462484 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:04.462419    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:11:04.462540 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:04.462483    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs podName:0238389f-6661-453f-ba7b-b33a3c34271c nodeName:}" failed. No retries permitted until 2026-09-09 12:11:12.462463179 +0000 UTC m=+17.104082364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs") pod "network-metrics-daemon-dj698" (UID: "0238389f-6661-453f-ba7b-b33a3c34271c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:11:04.462892 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:04.462875    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:11:04.462943 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:04.462930    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret podName:a18b39f3-278e-4cc9-97b0-867a633220f3 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:12.462913778 +0000 UTC m=+17.104532961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret") pod "global-pull-secret-syncer-m6ssq" (UID: "a18b39f3-278e-4cc9-97b0-867a633220f3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:11:04.563475 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:04.562851    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wdj\" (UniqueName: \"kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj\") pod \"network-check-target-n8bz5\" (UID: \"cee4188b-7e72-4b33-bde9-d1b2a585d0b5\") " pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:04.563475 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:04.563063    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:11:04.563475 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:04.563088    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:11:04.563475 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:04.563100    2585 projected.go:196] Error preparing data for projected volume kube-api-access-b4wdj for pod openshift-network-diagnostics/network-check-target-n8bz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:11:04.563475 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:04.563154    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj podName:cee4188b-7e72-4b33-bde9-d1b2a585d0b5 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:12.563136461 +0000 UTC m=+17.204755667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4wdj" (UniqueName: "kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj") pod "network-check-target-n8bz5" (UID: "cee4188b-7e72-4b33-bde9-d1b2a585d0b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:11:04.893899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:04.893863    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:04.894080 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:04.893985    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:04.894346 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:04.894323    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:04.894430 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:04.894413    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:05.896803 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:05.896774    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:05.897200 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:05.896894    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:06.893897 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:06.893858    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:06.894073 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:06.893865    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:06.894073 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:06.894002    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:06.894157 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:06.894074    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:07.893591 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:07.893548    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:07.894147 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:07.893710    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:08.893893 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:08.893857    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:08.894374 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:08.893858    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:08.894374 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:08.893968    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:08.894374 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:08.894049    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:09.893609 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:09.893572    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:09.893799 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:09.893734    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:10.893146 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:10.893110    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:10.893583 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:10.893220    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:10.893583 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:10.893294    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:10.893583 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:10.893414    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:11.893378 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:11.893340    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:11.893809 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:11.893450    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:12.517769 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:12.517725    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:12.517963 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:12.517786    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:12.517963 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:12.517879    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:11:12.517963 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:12.517944    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:11:12.517963 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:12.517956    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret podName:a18b39f3-278e-4cc9-97b0-867a633220f3 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:28.517939464 +0000 UTC m=+33.159558665 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret") pod "global-pull-secret-syncer-m6ssq" (UID: "a18b39f3-278e-4cc9-97b0-867a633220f3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:11:12.518147 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:12.518000    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs podName:0238389f-6661-453f-ba7b-b33a3c34271c nodeName:}" failed. No retries permitted until 2026-09-09 12:11:28.517983795 +0000 UTC m=+33.159602997 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs") pod "network-metrics-daemon-dj698" (UID: "0238389f-6661-453f-ba7b-b33a3c34271c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:11:12.618648 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:12.618613    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wdj\" (UniqueName: \"kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj\") pod \"network-check-target-n8bz5\" (UID: \"cee4188b-7e72-4b33-bde9-d1b2a585d0b5\") " pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:12.618841 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:12.618824    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:11:12.618904 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:12.618849    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:11:12.618904 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:12.618864    2585 projected.go:196] Error preparing data for projected volume kube-api-access-b4wdj for pod openshift-network-diagnostics/network-check-target-n8bz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:11:12.619004 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:12.618926    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj podName:cee4188b-7e72-4b33-bde9-d1b2a585d0b5 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:28.618908159 +0000 UTC m=+33.260527360 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4wdj" (UniqueName: "kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj") pod "network-check-target-n8bz5" (UID: "cee4188b-7e72-4b33-bde9-d1b2a585d0b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:11:12.893987 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:12.893915    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:12.894448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:12.893915    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:12.894448 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:12.894044    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:12.894448 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:12.894111    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:13.893437 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:13.893403    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:13.893756 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:13.893588    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:14.893297 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:14.893251    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:14.893805 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:14.893256    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:14.893805 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:14.893406    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:14.893805 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:14.893494    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:15.893916 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:15.893879    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:15.894262 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:15.893973    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:16.893738 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.893415    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:16.893861 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.893464    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:16.893861 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:16.893827    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:16.893944 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:16.893898    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:16.970962 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.970933    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-99k5f" event={"ID":"6499166f-4e4d-4db6-a1c7-5ef8a9502054","Type":"ContainerStarted","Data":"49dc4ad86ecb32fcaaef37b5014941ca69f583c70bae688a7d7d86bddb27c592"}
Sep 09 12:11:16.972237 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.972205    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vt2db" event={"ID":"9ce58793-93d8-4b04-af82-66521edd67dc","Type":"ContainerStarted","Data":"84312b7585ebb2c6752567611ef4c0b8931e68e370f92a4bfaf042229f1355d2"}
Sep 09 12:11:16.973530 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.973512    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hk4nf" event={"ID":"3b727a14-210f-4cfa-afde-83c1813bad01","Type":"ContainerStarted","Data":"e06ce0a51053b9633a795f1769f6d5482f5218f06f14c5b83c1d4e00aae04f05"}
Sep 09 12:11:16.975002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.974982    2585 generic.go:356] "Generic (PLEG): container finished" podID="5a64c743-7066-4939-9f91-88da5ad84d37" containerID="d0eb00d5ae8542ae748d0915c4b3d4bec65167e802a9f7472c6f235dedc7e7fc" exitCode=0
Sep 09 12:11:16.975075 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.975049    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wc5n" event={"ID":"5a64c743-7066-4939-9f91-88da5ad84d37","Type":"ContainerDied","Data":"d0eb00d5ae8542ae748d0915c4b3d4bec65167e802a9f7472c6f235dedc7e7fc"}
Sep 09 12:11:16.976642 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.976619    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r" event={"ID":"13374047-806d-4ebf-966f-56beae8c6ba5","Type":"ContainerStarted","Data":"c7daf2dd9b75da3f43ffc00731816914f9ef1f911ccda36f413cf2df872bfbb2"}
Sep 09 12:11:16.978292 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.978272    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7d845" event={"ID":"72f2bf19-006c-41ff-9f3e-e41c1d113274","Type":"ContainerStarted","Data":"2bfad472a8c620eb09c7325812f54c4189cecbc07a81b545861d75c869e3a9e2"}
Sep 09 12:11:16.980890 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.980869    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9" event={"ID":"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21","Type":"ContainerStarted","Data":"c26a548fc72121fff479b7d4856b1d0350afc11173006e6a9f86d8291a7ec034"}
Sep 09 12:11:16.980972 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.980928    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9" event={"ID":"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21","Type":"ContainerStarted","Data":"10b066dc8c740603e2b71dba655c31bb5444f57404c5bbc3b05147d29418e821"}
Sep 09 12:11:16.980972 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.980962    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9" event={"ID":"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21","Type":"ContainerStarted","Data":"77947c43f88eab320c92ac979d435d3f6bf9288432ea75a0ac077b4ff2a1cb1f"}
Sep 09 12:11:16.981054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.981005    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9" event={"ID":"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21","Type":"ContainerStarted","Data":"663cdcd4d431f46d704aef7ee13cb24ee09e9a1a3c59d7d34caafb74a4939987"}
Sep 09 12:11:16.981054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.981040    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9" event={"ID":"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21","Type":"ContainerStarted","Data":"d2c938e0c7702b8b2f8c4eb8bf33e06c6de6fb84c6ca732df8eb0ac97c68d943"}
Sep 09 12:11:16.984265 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.984244    2585 generic.go:356] "Generic (PLEG): container finished" podID="b793b7a48d2b0fa6c9c023903323e75e" containerID="5a1cef1a51fa7216be82a2b6fc79d7bd430e7012a7ece599805e174ba50a3387" exitCode=0
Sep 09 12:11:16.984362 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.984323    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal" event={"ID":"b793b7a48d2b0fa6c9c023903323e75e","Type":"ContainerDied","Data":"5a1cef1a51fa7216be82a2b6fc79d7bd430e7012a7ece599805e174ba50a3387"}
Sep 09 12:11:16.986387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.986362    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jwlkw" event={"ID":"af64da4a-c9ad-4944-bef3-583521578d0e","Type":"ContainerStarted","Data":"341eee57254304249a48421e871e994aefb04a5819d4e235e804426ad04b91bf"}
Sep 09 12:11:16.987678 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:16.987661    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal" event={"ID":"ba7b0232134df51badacb252b3d6af22","Type":"ContainerStarted","Data":"1e254a80370be98c008ce38362e050a56c1e8fbc870bc4ba3c5290bb7be5a2b7"}
Sep 09 12:11:17.003777 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:17.003741    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-99k5f" podStartSLOduration=2.007165821 podStartE2EDuration="21.003728436s" totalImagesPullingTime="18.996562615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:16.988781712 +0000 UTC m=+21.630400915" watchObservedRunningTime="2026-09-09 12:11:17.003728436 +0000 UTC m=+21.645347639"
Sep 09 12:11:17.003867 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:17.003830    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7d845" podStartSLOduration=1.891994038 podStartE2EDuration="21.003826393s" totalImagesPullingTime="19.111832355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-09 12:11:17.003414004 +0000 UTC m=+21.645033209" watchObservedRunningTime="2026-09-09 12:11:17.003826393 +0000 UTC m=+21.645445597"
Sep 09 12:11:17.022821 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:17.022776    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-207.ec2.internal" podStartSLOduration=21.022762415 podStartE2EDuration="21.022762415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:17.022254813 +0000 UTC m=+21.663874028" watchObservedRunningTime="2026-09-09 12:11:17.022762415 +0000 UTC m=+21.664381619"
Sep 09 12:11:17.068340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:17.068281    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jwlkw" podStartSLOduration=2.059485028 podStartE2EDuration="21.068261068s" totalImagesPullingTime="19.00877604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:17.041901813 +0000 UTC m=+21.683521018" watchObservedRunningTime="2026-09-09 12:11:17.068261068 +0000 UTC m=+21.709880275"
Sep 09 12:11:17.132931 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:17.132880    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hk4nf" podStartSLOduration=1.915135187 podStartE2EDuration="21.132862512s" totalImagesPullingTime="19.217727325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:17.112105679 +0000 UTC m=+21.753724885" watchObservedRunningTime="2026-09-09 12:11:17.132862512 +0000 UTC m=+21.774481712"
Sep 09 12:11:17.133104 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:17.133084    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vt2db" podStartSLOduration=2.185497989 podStartE2EDuration="21.133078671s" totalImagesPullingTime="18.947580682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:17.132852944 +0000 UTC m=+21.774472151" watchObservedRunningTime="2026-09-09 12:11:17.133078671 +0000 UTC m=+21.774697875"
Sep 09 12:11:17.893124 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:17.893093    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:17.893326 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:17.893196    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:17.990600 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:17.990512    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-49nz8" event={"ID":"243df091-5947-47d0-bd58-fd2dd2bfe015","Type":"ContainerStarted","Data":"77dd48d5c4531b17cb7ce137fdd89aa1c5ca40e0847ecc32e986204a297a4600"}
Sep 09 12:11:17.993459 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:17.993437    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9" event={"ID":"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21","Type":"ContainerStarted","Data":"40946dfc0e6041a5f2070b491da3513ec05f7c0c1ac0b169367081a80c9d7c5d"}
Sep 09 12:11:17.995200 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:17.995157    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal" event={"ID":"b793b7a48d2b0fa6c9c023903323e75e","Type":"ContainerStarted","Data":"5788c3d0d626deef89b9e2fa3cfed25d14a4719afb457e05180f5a80a841366b"}
Sep 09 12:11:18.002224 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:18.002206    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:11:18.002724 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:18.002690    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:11:18.022192 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:18.022154    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-49nz8" podStartSLOduration=2.972957745 podStartE2EDuration="22.022141271s" totalImagesPullingTime="19.049183526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:18.006019505 +0000 UTC m=+22.647638710" watchObservedRunningTime="2026-09-09 12:11:18.022141271 +0000 UTC m=+22.663760475"
Sep 09 12:11:18.037635 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:18.037601    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-207.ec2.internal" podStartSLOduration=22.037593392 podStartE2EDuration="22.037593392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:18.037265232 +0000 UTC m=+22.678884436" watchObservedRunningTime="2026-09-09 12:11:18.037593392 +0000 UTC m=+22.679212596"
Sep 09 12:11:18.136635 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:18.136506    2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 12:11:18.858525 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:18.858297    2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T12:11:18.136522684Z","UUID":"a77540f5-a882-4ce4-81a0-839716b77171","Handler":null,"Name":"","Endpoint":""}
Sep 09 12:11:18.860323 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:18.860301    2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 12:11:18.860481 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:18.860337    2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 12:11:18.893544 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:18.893516    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:18.893687 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:18.893611    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:18.893687 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:18.893660    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:18.893788 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:18.893750    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:19.002450 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:19.002412    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r" event={"ID":"13374047-806d-4ebf-966f-56beae8c6ba5","Type":"ContainerStarted","Data":"19838cf47d89e751f592e9c8f49245ad15f8880f1e3cd0be71bdcc1ca62e0346"}
Sep 09 12:11:19.893026 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:19.893001    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:19.893160 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:19.893133    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:20.007414 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:20.007372    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r" event={"ID":"13374047-806d-4ebf-966f-56beae8c6ba5","Type":"ContainerStarted","Data":"cad2b5fdfba537feb65b03d9a9c5e294a98120399486608ffa118c1549e7e03c"}
Sep 09 12:11:20.011212 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:20.011184    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9" event={"ID":"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21","Type":"ContainerStarted","Data":"46adf912a57bea794a4fd7e4bf17d2f1e9b8dafcafa442d69f7d29651083cc7f"}
Sep 09 12:11:20.011333 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:20.011198    2585 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 12:11:20.893296 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:20.893263    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:20.893486 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:20.893280    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:20.893486 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:20.893384    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:20.893486 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:20.893463    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:21.893983 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:21.893944    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:21.894391 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:21.894105    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:22.020591 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:22.020554    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9" event={"ID":"fc9f3a1d-ab68-49a3-b267-5cc0113fcc21","Type":"ContainerStarted","Data":"9bc3c98de859e5cb656365b3e74d9ca80e5b572b310f3853475e10560a5fe880"}
Sep 09 12:11:22.020903 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:22.020883    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:11:22.021008 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:22.020970    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:11:22.038690 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:22.038664    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:11:22.080256 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:22.080196    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvj9r" podStartSLOduration=4.139586678 podStartE2EDuration="26.080161361s" totalImagesPullingTime="21.940574683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:20.039692909 +0000 UTC m=+24.681312113" watchObservedRunningTime="2026-09-09 12:11:22.080161361 +0000 UTC m=+26.721780567"
Sep 09 12:11:22.135438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:22.135380    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9" podStartSLOduration=7.034789876 podStartE2EDuration="26.135359206s" totalImagesPullingTime="19.10056933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:22.080621448 +0000 UTC m=+26.722240652" watchObservedRunningTime="2026-09-09 12:11:22.135359206 +0000 UTC m=+26.776978412"
Sep 09 12:11:22.894063 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:22.893856    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:22.894063 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:22.893865    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:22.894453 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:22.894221    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:22.894531 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:22.894512    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:23.023191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.023157    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:11:23.038618 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.038589    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:11:23.075187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.075160    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:11:23.075340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.075287    2585 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 12:11:23.075742 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.075719    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vt2db"
Sep 09 12:11:23.341017 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.340987    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dj698"]
Sep 09 12:11:23.341207 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.341153    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:23.341302 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:23.341280    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:23.351476 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.351446    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n8bz5"]
Sep 09 12:11:23.351640 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.351558    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:23.351720 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:23.351662    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:23.355456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.355430    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-m6ssq"]
Sep 09 12:11:23.355591 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:23.355512    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:23.355653 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:23.355594    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:24.026860 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:24.026824    2585 generic.go:356] "Generic (PLEG): container finished" podID="5a64c743-7066-4939-9f91-88da5ad84d37" containerID="2b2f324b3c196f68a2d4ff24b3c95ae22f72768cf3f0dbaa7582195042bb60f3" exitCode=0
Sep 09 12:11:24.027393 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:24.026911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wc5n" event={"ID":"5a64c743-7066-4939-9f91-88da5ad84d37","Type":"ContainerDied","Data":"2b2f324b3c196f68a2d4ff24b3c95ae22f72768cf3f0dbaa7582195042bb60f3"}
Sep 09 12:11:24.893713 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:24.893675    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:24.893898 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:24.893717    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:24.893898 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:24.893807    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:24.893898 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:24.893828    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:24.893898 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:24.893887    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:24.894119 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:24.893996    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:26.033198 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:26.033163    2585 generic.go:356] "Generic (PLEG): container finished" podID="5a64c743-7066-4939-9f91-88da5ad84d37" containerID="14523b818e41bafd4edf8f62b38b88b20a8620e1e77ea89205982d66baf32128" exitCode=0
Sep 09 12:11:26.033741 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:26.033236    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wc5n" event={"ID":"5a64c743-7066-4939-9f91-88da5ad84d37","Type":"ContainerDied","Data":"14523b818e41bafd4edf8f62b38b88b20a8620e1e77ea89205982d66baf32128"}
Sep 09 12:11:26.893973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:26.893756    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:26.894145 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:26.893793    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:26.894145 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:26.894085    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:26.894265 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:26.894186    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:26.894265 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:26.893796    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:26.894374 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:26.894303    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:28.043130 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:28.043097    2585 generic.go:356] "Generic (PLEG): container finished" podID="5a64c743-7066-4939-9f91-88da5ad84d37" containerID="118b90b8913211b074b7e6a2b8ff3493ee9925ef07ed29cd35baa771ffede44b" exitCode=0
Sep 09 12:11:28.043619 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:28.043157    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wc5n" event={"ID":"5a64c743-7066-4939-9f91-88da5ad84d37","Type":"ContainerDied","Data":"118b90b8913211b074b7e6a2b8ff3493ee9925ef07ed29cd35baa771ffede44b"}
Sep 09 12:11:28.529740 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:28.529684    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:28.529931 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:28.529753    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:28.529931 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.529853    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:11:28.529931 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.529856    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:11:28.529931 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.529910    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs podName:0238389f-6661-453f-ba7b-b33a3c34271c nodeName:}" failed. No retries permitted until 2026-09-09 12:12:00.529894171 +0000 UTC m=+65.171513354 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs") pod "network-metrics-daemon-dj698" (UID: "0238389f-6661-453f-ba7b-b33a3c34271c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:11:28.529931 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.529927    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret podName:a18b39f3-278e-4cc9-97b0-867a633220f3 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:00.529919508 +0000 UTC m=+65.171538692 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret") pod "global-pull-secret-syncer-m6ssq" (UID: "a18b39f3-278e-4cc9-97b0-867a633220f3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:11:28.630771 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:28.630725    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wdj\" (UniqueName: \"kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj\") pod \"network-check-target-n8bz5\" (UID: \"cee4188b-7e72-4b33-bde9-d1b2a585d0b5\") " pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:28.630944 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.630874    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:11:28.630944 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.630893    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:11:28.630944 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.630903    2585 projected.go:196] Error preparing data for projected volume kube-api-access-b4wdj for pod openshift-network-diagnostics/network-check-target-n8bz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:11:28.631049 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.630955    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj podName:cee4188b-7e72-4b33-bde9-d1b2a585d0b5 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:00.630941819 +0000 UTC m=+65.272561004 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4wdj" (UniqueName: "kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj") pod "network-check-target-n8bz5" (UID: "cee4188b-7e72-4b33-bde9-d1b2a585d0b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:11:28.893231 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:28.893207    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:28.893330 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:28.893301    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:28.893413 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.893396    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dj698" podUID="0238389f-6661-453f-ba7b-b33a3c34271c"
Sep 09 12:11:28.893448 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.893304    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n8bz5" podUID="cee4188b-7e72-4b33-bde9-d1b2a585d0b5"
Sep 09 12:11:28.893448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:28.893308    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:28.893508 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:28.893477    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m6ssq" podUID="a18b39f3-278e-4cc9-97b0-867a633220f3"
Sep 09 12:11:29.161608 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.161537    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-207.ec2.internal" event="NodeReady"
Sep 09 12:11:29.161989 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.161666    2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 12:11:29.231450 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.231418    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-785748674c-9n6ft"]
Sep 09 12:11:29.243002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.242978    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.248084 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.248063    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 12:11:29.248084 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.248082    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 12:11:29.248534 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.248515    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bkd5d\""
Sep 09 12:11:29.248632 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.248595    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 12:11:29.257433 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.257411    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 12:11:29.262967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.262947    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gdzzk"]
Sep 09 12:11:29.279440 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.279411    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-785748674c-9n6ft"]
Sep 09 12:11:29.279536 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.279460    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gdzzk"
Sep 09 12:11:29.297799 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.297780    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 12:11:29.298069 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.298054    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xqhh6\""
Sep 09 12:11:29.298146 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.298131    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 12:11:29.298679 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.298658    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 12:11:29.310968 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.310942    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gdzzk"]
Sep 09 12:11:29.334887 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.334863    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.334988 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.334894    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-installation-pull-secrets\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.334988 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.334911    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-bound-sa-token\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.334988 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.334928    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-image-registry-private-configuration\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.334988 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.334942    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vntd\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-kube-api-access-8vntd\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.335187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.335014    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-registry-certificates\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.335187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.335076    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1efd99b8-2516-4b3f-8f2f-39cfa33c9c58-cert\") pod \"ingress-canary-gdzzk\" (UID: \"1efd99b8-2516-4b3f-8f2f-39cfa33c9c58\") " pod="openshift-ingress-canary/ingress-canary-gdzzk"
Sep 09 12:11:29.335187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.335162    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzcqm\" (UniqueName: \"kubernetes.io/projected/1efd99b8-2516-4b3f-8f2f-39cfa33c9c58-kube-api-access-zzcqm\") pod \"ingress-canary-gdzzk\" (UID: \"1efd99b8-2516-4b3f-8f2f-39cfa33c9c58\") " pod="openshift-ingress-canary/ingress-canary-gdzzk"
Sep 09 12:11:29.335331 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.335235    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6955fe65-c931-4735-8315-03f05ae37092-ca-trust-extracted\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.335331 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.335259    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-trusted-ca\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.377797 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.377769    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vrrhx"]
Sep 09 12:11:29.387665 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.387643    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.393775 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.392591    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 12:11:29.393775 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.393123    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 12:11:29.393775 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.393211    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mksrl\""
Sep 09 12:11:29.393775 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.393658    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vrrhx"]
Sep 09 12:11:29.435656 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.435571    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzcqm\" (UniqueName: \"kubernetes.io/projected/1efd99b8-2516-4b3f-8f2f-39cfa33c9c58-kube-api-access-zzcqm\") pod \"ingress-canary-gdzzk\" (UID: \"1efd99b8-2516-4b3f-8f2f-39cfa33c9c58\") " pod="openshift-ingress-canary/ingress-canary-gdzzk"
Sep 09 12:11:29.435656 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.435620    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46998610-68a0-4431-bc25-313e17abd0ad-tmp-dir\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.435890 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.435736    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv8v8\" (UniqueName: \"kubernetes.io/projected/46998610-68a0-4431-bc25-313e17abd0ad-kube-api-access-qv8v8\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.435890 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.435786    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6955fe65-c931-4735-8315-03f05ae37092-ca-trust-extracted\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.435890 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.435803    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-trusted-ca\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.435890 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.435830    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.435890 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.435845    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.435890 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.435864    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-installation-pull-secrets\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.435890 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.435889    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-bound-sa-token\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.436233 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.435947    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-image-registry-private-configuration\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.436233 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:29.436003    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:11:29.436233 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:29.436022    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785748674c-9n6ft: secret "image-registry-tls" not found
Sep 09 12:11:29.436233 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.436060    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vntd\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-kube-api-access-8vntd\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.436233 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:29.436087    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls podName:6955fe65-c931-4735-8315-03f05ae37092 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:29.936066116 +0000 UTC m=+34.577685324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls") pod "image-registry-785748674c-9n6ft" (UID: "6955fe65-c931-4735-8315-03f05ae37092") : secret "image-registry-tls" not found
Sep 09 12:11:29.436233 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.436159    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-registry-certificates\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.436233 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.436193    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6955fe65-c931-4735-8315-03f05ae37092-ca-trust-extracted\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.436233 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.436198    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46998610-68a0-4431-bc25-313e17abd0ad-config-volume\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.436676 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.436255    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1efd99b8-2516-4b3f-8f2f-39cfa33c9c58-cert\") pod \"ingress-canary-gdzzk\" (UID: \"1efd99b8-2516-4b3f-8f2f-39cfa33c9c58\") " pod="openshift-ingress-canary/ingress-canary-gdzzk"
Sep 09 12:11:29.436773 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.436751    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-registry-certificates\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.436840 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.436819    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-trusted-ca\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.439917 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.439894    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-installation-pull-secrets\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.440018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.439894    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-image-registry-private-configuration\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.440018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.439967    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1efd99b8-2516-4b3f-8f2f-39cfa33c9c58-cert\") pod \"ingress-canary-gdzzk\" (UID: \"1efd99b8-2516-4b3f-8f2f-39cfa33c9c58\") " pod="openshift-ingress-canary/ingress-canary-gdzzk"
Sep 09 12:11:29.447090 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.447063    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-bound-sa-token\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.447204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.447184    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzcqm\" (UniqueName: \"kubernetes.io/projected/1efd99b8-2516-4b3f-8f2f-39cfa33c9c58-kube-api-access-zzcqm\") pod \"ingress-canary-gdzzk\" (UID: \"1efd99b8-2516-4b3f-8f2f-39cfa33c9c58\") " pod="openshift-ingress-canary/ingress-canary-gdzzk"
Sep 09 12:11:29.448441 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.448423    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vntd\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-kube-api-access-8vntd\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.537623 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.537585    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.537830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.537635    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46998610-68a0-4431-bc25-313e17abd0ad-config-volume\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.537830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.537663    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46998610-68a0-4431-bc25-313e17abd0ad-tmp-dir\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.537830 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:29.537771    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:11:29.537830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.537779    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8v8\" (UniqueName: \"kubernetes.io/projected/46998610-68a0-4431-bc25-313e17abd0ad-kube-api-access-qv8v8\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.538010 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:29.537847    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls podName:46998610-68a0-4431-bc25-313e17abd0ad nodeName:}" failed. No retries permitted until 2026-09-09 12:11:30.03782532 +0000 UTC m=+34.679444528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls") pod "dns-default-vrrhx" (UID: "46998610-68a0-4431-bc25-313e17abd0ad") : secret "dns-default-metrics-tls" not found
Sep 09 12:11:29.538010 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.537971    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46998610-68a0-4431-bc25-313e17abd0ad-tmp-dir\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.538180 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.538162    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46998610-68a0-4431-bc25-313e17abd0ad-config-volume\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.548418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.548387    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv8v8\" (UniqueName: \"kubernetes.io/projected/46998610-68a0-4431-bc25-313e17abd0ad-kube-api-access-qv8v8\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:29.587338 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.587302    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gdzzk"
Sep 09 12:11:29.892362 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.892324    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gdzzk"]
Sep 09 12:11:29.896306 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:11:29.896280    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1efd99b8_2516_4b3f_8f2f_39cfa33c9c58.slice/crio-2d2b74673d1581f045a00feaf74486ed2d5fb84248690990cf047f46355849dd WatchSource:0}: Error finding container 2d2b74673d1581f045a00feaf74486ed2d5fb84248690990cf047f46355849dd: Status 404 returned error can't find the container with id 2d2b74673d1581f045a00feaf74486ed2d5fb84248690990cf047f46355849dd
Sep 09 12:11:29.940847 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:29.940811    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:29.941001 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:29.940966    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:11:29.941001 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:29.940985    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785748674c-9n6ft: secret "image-registry-tls" not found
Sep 09 12:11:29.941081 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:29.941037    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls podName:6955fe65-c931-4735-8315-03f05ae37092 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:30.941021988 +0000 UTC m=+35.582641169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls") pod "image-registry-785748674c-9n6ft" (UID: "6955fe65-c931-4735-8315-03f05ae37092") : secret "image-registry-tls" not found
Sep 09 12:11:30.042236 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.042199    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:30.042371 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:30.042347    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:11:30.042417 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:30.042408    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls podName:46998610-68a0-4431-bc25-313e17abd0ad nodeName:}" failed. No retries permitted until 2026-09-09 12:11:31.042392134 +0000 UTC m=+35.684011317 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls") pod "dns-default-vrrhx" (UID: "46998610-68a0-4431-bc25-313e17abd0ad") : secret "dns-default-metrics-tls" not found
Sep 09 12:11:30.049124 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.049091    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gdzzk" event={"ID":"1efd99b8-2516-4b3f-8f2f-39cfa33c9c58","Type":"ContainerStarted","Data":"2d2b74673d1581f045a00feaf74486ed2d5fb84248690990cf047f46355849dd"}
Sep 09 12:11:30.894034 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.893998    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:11:30.894486 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.893998    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:11:30.894486 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.894149    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:11:30.898110 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.898082    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:11:30.898110 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.898095    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g5zgt\""
Sep 09 12:11:30.898316 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.898296    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:11:30.898359 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.898335    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:11:30.898581 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.898566    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:11:30.898801 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.898782    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6s86q\""
Sep 09 12:11:30.951919 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:30.951887    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:30.952107 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:30.952053    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:11:30.952107 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:30.952072    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785748674c-9n6ft: secret "image-registry-tls" not found
Sep 09 12:11:30.952217 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:30.952136    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls podName:6955fe65-c931-4735-8315-03f05ae37092 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:32.952119647 +0000 UTC m=+37.593738830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls") pod "image-registry-785748674c-9n6ft" (UID: "6955fe65-c931-4735-8315-03f05ae37092") : secret "image-registry-tls" not found
Sep 09 12:11:31.052922 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:31.052878    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:31.053106 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:31.053003    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:11:31.053106 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:31.053049    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls podName:46998610-68a0-4431-bc25-313e17abd0ad nodeName:}" failed. No retries permitted until 2026-09-09 12:11:33.053036495 +0000 UTC m=+37.694655676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls") pod "dns-default-vrrhx" (UID: "46998610-68a0-4431-bc25-313e17abd0ad") : secret "dns-default-metrics-tls" not found
Sep 09 12:11:32.970177 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:32.969994    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:32.970177 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:32.970172    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:11:32.970632 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:32.970191    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785748674c-9n6ft: secret "image-registry-tls" not found
Sep 09 12:11:32.970632 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:32.970249    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls podName:6955fe65-c931-4735-8315-03f05ae37092 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:36.970229897 +0000 UTC m=+41.611849102 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls") pod "image-registry-785748674c-9n6ft" (UID: "6955fe65-c931-4735-8315-03f05ae37092") : secret "image-registry-tls" not found
Sep 09 12:11:33.059825 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:33.059791    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gdzzk" event={"ID":"1efd99b8-2516-4b3f-8f2f-39cfa33c9c58","Type":"ContainerStarted","Data":"7f399db0bcabc60bc78e5a09e6356a10544134b674a4575f01c6ad1dfd54b7b7"}
Sep 09 12:11:33.071308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:33.071269    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:33.071442 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:33.071383    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:11:33.071442 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:33.071436    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls podName:46998610-68a0-4431-bc25-313e17abd0ad nodeName:}" failed. No retries permitted until 2026-09-09 12:11:37.07142258 +0000 UTC m=+41.713041782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls") pod "dns-default-vrrhx" (UID: "46998610-68a0-4431-bc25-313e17abd0ad") : secret "dns-default-metrics-tls" not found
Sep 09 12:11:33.079190 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:33.079135    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gdzzk" podStartSLOduration=1.127143361 podStartE2EDuration="4.079113681s" totalImagesPullingTime="2.95197032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:33.07803841 +0000 UTC m=+37.719657626" watchObservedRunningTime="2026-09-09 12:11:33.079113681 +0000 UTC m=+37.720732879"
Sep 09 12:11:34.830640 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:34.830583    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vt2db_9ce58793-93d8-4b04-af82-66521edd67dc/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:34.925721 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:34.925665    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vt2db_9ce58793-93d8-4b04-af82-66521edd67dc/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:36.069951 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:36.069919    2585 generic.go:356] "Generic (PLEG): container finished" podID="5a64c743-7066-4939-9f91-88da5ad84d37" containerID="c30a8b3c26bbef82d52f37c6d8978ab80c7df00f5c80d03fa7410937c5977183" exitCode=0
Sep 09 12:11:36.070311 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:36.069988    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wc5n" event={"ID":"5a64c743-7066-4939-9f91-88da5ad84d37","Type":"ContainerDied","Data":"c30a8b3c26bbef82d52f37c6d8978ab80c7df00f5c80d03fa7410937c5977183"}
Sep 09 12:11:37.004643 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:37.004599    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:37.004824 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:37.004772    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:11:37.004824 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:37.004797    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785748674c-9n6ft: secret "image-registry-tls" not found
Sep 09 12:11:37.004891 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:37.004851    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls podName:6955fe65-c931-4735-8315-03f05ae37092 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:45.0048369 +0000 UTC m=+49.646456081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls") pod "image-registry-785748674c-9n6ft" (UID: "6955fe65-c931-4735-8315-03f05ae37092") : secret "image-registry-tls" not found
Sep 09 12:11:37.075788 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:37.075749    2585 generic.go:356] "Generic (PLEG): container finished" podID="5a64c743-7066-4939-9f91-88da5ad84d37" containerID="2e5b4155aa1fd5300ec29f8242bc1d329b53a44a77622636c43fe2d3d2f49dd0" exitCode=0
Sep 09 12:11:37.076233 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:37.075799    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wc5n" event={"ID":"5a64c743-7066-4939-9f91-88da5ad84d37","Type":"ContainerDied","Data":"2e5b4155aa1fd5300ec29f8242bc1d329b53a44a77622636c43fe2d3d2f49dd0"}
Sep 09 12:11:37.105111 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:37.105084    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:37.105262 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:37.105243    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:11:37.105317 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:37.105307    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls podName:46998610-68a0-4431-bc25-313e17abd0ad nodeName:}" failed. No retries permitted until 2026-09-09 12:11:45.105290454 +0000 UTC m=+49.746909637 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls") pod "dns-default-vrrhx" (UID: "46998610-68a0-4431-bc25-313e17abd0ad") : secret "dns-default-metrics-tls" not found
Sep 09 12:11:38.081088 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:38.081054    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wc5n" event={"ID":"5a64c743-7066-4939-9f91-88da5ad84d37","Type":"ContainerStarted","Data":"0ccb1753d9f32e35fb5c156f5a0012a9cc14dd9ad1bea40af86d07cb6272478c"}
Sep 09 12:11:38.103478 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:38.103430    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7wc5n" podStartSLOduration=6.879157391 podStartE2EDuration="42.103413476s" totalImagesPullingTime="35.224256085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:11:38.103384477 +0000 UTC m=+42.745003681" watchObservedRunningTime="2026-09-09 12:11:38.103413476 +0000 UTC m=+42.745032677"
Sep 09 12:11:38.229742 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:38.229688    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vt2db_9ce58793-93d8-4b04-af82-66521edd67dc/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:38.330644 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:38.330616    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vt2db_9ce58793-93d8-4b04-af82-66521edd67dc/konnectivity-agent/0.log" limit=1024
Sep 09 12:11:45.062639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:45.062583    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:11:45.063054 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:45.062757    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:11:45.063054 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:45.062779    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785748674c-9n6ft: secret "image-registry-tls" not found
Sep 09 12:11:45.063054 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:45.062847    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls podName:6955fe65-c931-4735-8315-03f05ae37092 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:01.062831542 +0000 UTC m=+65.704450724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls") pod "image-registry-785748674c-9n6ft" (UID: "6955fe65-c931-4735-8315-03f05ae37092") : secret "image-registry-tls" not found
Sep 09 12:11:45.163873 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:45.163820    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:11:45.164086 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:45.163996    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:11:45.164158 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:45.164092    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls podName:46998610-68a0-4431-bc25-313e17abd0ad nodeName:}" failed. No retries permitted until 2026-09-09 12:12:01.164070743 +0000 UTC m=+65.805689942 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls") pod "dns-default-vrrhx" (UID: "46998610-68a0-4431-bc25-313e17abd0ad") : secret "dns-default-metrics-tls" not found
Sep 09 12:11:55.048674 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:55.048638    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j99x9"
Sep 09 12:11:58.664889 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.664847    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8vfpn"]
Sep 09 12:11:58.668182 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.668160    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.670646 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.670625    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 12:11:58.670646 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.670628    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 12:11:58.670848 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.670632    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lrqz2\""
Sep 09 12:11:58.671494 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.671478    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 12:11:58.671541 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.671518    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 12:11:58.686666 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.686646    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8vfpn"]
Sep 09 12:11:58.767399 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.767358    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r79qs\" (UniqueName: \"kubernetes.io/projected/095003f4-af63-4eda-82af-671ce6a44404-kube-api-access-r79qs\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.767591 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.767501    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.767591 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.767538    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/095003f4-af63-4eda-82af-671ce6a44404-data-volume\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.767591 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.767558    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/095003f4-af63-4eda-82af-671ce6a44404-crio-socket\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.767591 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.767575    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.868735 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.868661    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/095003f4-af63-4eda-82af-671ce6a44404-data-volume\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.868877 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.868762    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/095003f4-af63-4eda-82af-671ce6a44404-crio-socket\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.868877 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.868804    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.868877 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.868870    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r79qs\" (UniqueName: \"kubernetes.io/projected/095003f4-af63-4eda-82af-671ce6a44404-kube-api-access-r79qs\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.869020 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.869009    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.869151 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:58.869132    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:11:58.869233 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:58.869221    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:59.369192924 +0000 UTC m=+64.010812124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : configmap "kube-rbac-proxy" not found
Sep 09 12:11:58.869869 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.869844    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/095003f4-af63-4eda-82af-671ce6a44404-data-volume\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.869968 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.869938    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/095003f4-af63-4eda-82af-671ce6a44404-crio-socket\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:58.870057 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:58.870043    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:11:58.870105 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:58.870100    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:11:59.370084783 +0000 UTC m=+64.011703972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:11:58.880754 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:58.880728    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r79qs\" (UniqueName: \"kubernetes.io/projected/095003f4-af63-4eda-82af-671ce6a44404-kube-api-access-r79qs\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:59.373131 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:59.373086    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:59.373328 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:11:59.373139    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:11:59.373328 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:59.373200    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:11:59.373328 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:59.373273    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:00.373257031 +0000 UTC m=+65.014876212 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : configmap "kube-rbac-proxy" not found
Sep 09 12:11:59.373328 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:59.373279    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:11:59.373328 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:11:59.373326    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:00.373312043 +0000 UTC m=+65.014931225 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:12:00.381734 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.381664    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:00.382135 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.381753    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:00.382135 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:00.381797    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:12:00.382135 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:00.381883    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:02.381864182 +0000 UTC m=+67.023483560 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : configmap "kube-rbac-proxy" not found
Sep 09 12:12:00.382135 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:00.381896    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:12:00.382135 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:00.381955    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:02.381942759 +0000 UTC m=+67.023561941 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:12:00.437391 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.437364    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7d845_72f2bf19-006c-41ff-9f3e-e41c1d113274/dns-node-resolver/0.log"
Sep 09 12:12:00.583299 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.583257    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:12:00.583471 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.583358    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:12:00.585679 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.585659    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:12:00.586478 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.586464    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:12:00.593507 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:00.593490    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 12:12:00.593564 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:00.593549    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs podName:0238389f-6661-453f-ba7b-b33a3c34271c nodeName:}" failed. No retries permitted until 2026-09-09 12:13:04.593533147 +0000 UTC m=+129.235152329 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs") pod "network-metrics-daemon-dj698" (UID: "0238389f-6661-453f-ba7b-b33a3c34271c") : secret "metrics-daemon-secret" not found
Sep 09 12:12:00.595826 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.595806    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a18b39f3-278e-4cc9-97b0-867a633220f3-original-pull-secret\") pod \"global-pull-secret-syncer-m6ssq\" (UID: \"a18b39f3-278e-4cc9-97b0-867a633220f3\") " pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:12:00.616049 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.616019    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m6ssq"
Sep 09 12:12:00.684363 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.684333    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wdj\" (UniqueName: \"kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj\") pod \"network-check-target-n8bz5\" (UID: \"cee4188b-7e72-4b33-bde9-d1b2a585d0b5\") " pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:12:00.687206 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.687186    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:12:00.698915 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.698892    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:12:00.708488 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.708467    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4wdj\" (UniqueName: \"kubernetes.io/projected/cee4188b-7e72-4b33-bde9-d1b2a585d0b5-kube-api-access-b4wdj\") pod \"network-check-target-n8bz5\" (UID: \"cee4188b-7e72-4b33-bde9-d1b2a585d0b5\") " pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:12:00.884553 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.884513    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-m6ssq"]
Sep 09 12:12:00.887881 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:00.887857    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18b39f3_278e_4cc9_97b0_867a633220f3.slice/crio-2e41d09df664b5ab1db1eb6a2c61042b019973abaafd4271882a1204057dc94f WatchSource:0}: Error finding container 2e41d09df664b5ab1db1eb6a2c61042b019973abaafd4271882a1204057dc94f: Status 404 returned error can't find the container with id 2e41d09df664b5ab1db1eb6a2c61042b019973abaafd4271882a1204057dc94f
Sep 09 12:12:00.922882 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.922818    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g5zgt\""
Sep 09 12:12:00.930978 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:00.930953    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:12:01.087287 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:01.087244    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:12:01.087460 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:01.087399    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:12:01.087460 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:01.087418    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785748674c-9n6ft: secret "image-registry-tls" not found
Sep 09 12:12:01.087576 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:01.087474    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls podName:6955fe65-c931-4735-8315-03f05ae37092 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:33.087456094 +0000 UTC m=+97.729075276 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls") pod "image-registry-785748674c-9n6ft" (UID: "6955fe65-c931-4735-8315-03f05ae37092") : secret "image-registry-tls" not found
Sep 09 12:12:01.135539 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:01.135506    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-m6ssq" event={"ID":"a18b39f3-278e-4cc9-97b0-867a633220f3","Type":"ContainerStarted","Data":"2e41d09df664b5ab1db1eb6a2c61042b019973abaafd4271882a1204057dc94f"}
Sep 09 12:12:01.188047 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:01.187973    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:12:01.188189 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:01.188124    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:12:01.188189 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:01.188186    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls podName:46998610-68a0-4431-bc25-313e17abd0ad nodeName:}" failed. No retries permitted until 2026-09-09 12:12:33.188171947 +0000 UTC m=+97.829791129 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls") pod "dns-default-vrrhx" (UID: "46998610-68a0-4431-bc25-313e17abd0ad") : secret "dns-default-metrics-tls" not found
Sep 09 12:12:01.200339 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:01.200313    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n8bz5"]
Sep 09 12:12:01.203738 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:01.203693    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee4188b_7e72_4b33_bde9_d1b2a585d0b5.slice/crio-58f739ed919c73fd01841400f01e3a953d3bec804438014c80875319342b724a WatchSource:0}: Error finding container 58f739ed919c73fd01841400f01e3a953d3bec804438014c80875319342b724a: Status 404 returned error can't find the container with id 58f739ed919c73fd01841400f01e3a953d3bec804438014c80875319342b724a
Sep 09 12:12:01.445473 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:01.445397    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jwlkw_af64da4a-c9ad-4944-bef3-583521578d0e/node-ca/0.log"
Sep 09 12:12:02.140301 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.140264    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n8bz5" event={"ID":"cee4188b-7e72-4b33-bde9-d1b2a585d0b5","Type":"ContainerStarted","Data":"58f739ed919c73fd01841400f01e3a953d3bec804438014c80875319342b724a"}
Sep 09 12:12:02.243815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.243782    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gdzzk_1efd99b8-2516-4b3f-8f2f-39cfa33c9c58/serve-healthcheck-canary/0.log"
Sep 09 12:12:02.343025 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.342993    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"]
Sep 09 12:12:02.346039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.346007    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:02.348367 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.348340    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 09 12:12:02.348505 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.348374    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 09 12:12:02.349099 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.349078    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k6bfc\""
Sep 09 12:12:02.349212 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.349077    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:12:02.356070 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.356043    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"]
Sep 09 12:12:02.397218 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.397137    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:02.397218 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.397180    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:02.397373 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:02.397272    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:12:02.397373 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:02.397359    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:12:02.397439 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:02.397362    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:06.397342431 +0000 UTC m=+71.038961613 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : configmap "kube-rbac-proxy" not found
Sep 09 12:12:02.397439 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:02.397431    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:06.397392594 +0000 UTC m=+71.039011775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:12:02.498457 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.498412    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:02.498457 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.498460    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84nxj\" (UniqueName: \"kubernetes.io/projected/ea9590d0-7506-42a0-84e8-e498a4df5b3d-kube-api-access-84nxj\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:02.598998 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.598963    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:02.599193 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.599010    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84nxj\" (UniqueName: \"kubernetes.io/projected/ea9590d0-7506-42a0-84e8-e498a4df5b3d-kube-api-access-84nxj\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:02.599193 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:02.599132    2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:12:02.599273 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:02.599207    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls podName:ea9590d0-7506-42a0-84e8-e498a4df5b3d nodeName:}" failed. No retries permitted until 2026-09-09 12:12:03.099188791 +0000 UTC m=+67.740807992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-l5kgg" (UID: "ea9590d0-7506-42a0-84e8-e498a4df5b3d") : secret "samples-operator-tls" not found
Sep 09 12:12:02.609193 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:02.609166    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84nxj\" (UniqueName: \"kubernetes.io/projected/ea9590d0-7506-42a0-84e8-e498a4df5b3d-kube-api-access-84nxj\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:03.104334 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:03.104284    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:03.104546 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:03.104458    2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:12:03.104610 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:03.104592    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls podName:ea9590d0-7506-42a0-84e8-e498a4df5b3d nodeName:}" failed. No retries permitted until 2026-09-09 12:12:04.104570589 +0000 UTC m=+68.746189789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-l5kgg" (UID: "ea9590d0-7506-42a0-84e8-e498a4df5b3d") : secret "samples-operator-tls" not found
Sep 09 12:12:04.113435 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:04.113396    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:04.113909 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:04.113539    2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:12:04.113909 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:04.113606    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls podName:ea9590d0-7506-42a0-84e8-e498a4df5b3d nodeName:}" failed. No retries permitted until 2026-09-09 12:12:06.113588488 +0000 UTC m=+70.755207685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-l5kgg" (UID: "ea9590d0-7506-42a0-84e8-e498a4df5b3d") : secret "samples-operator-tls" not found
Sep 09 12:12:05.151228 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:05.151137    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n8bz5" event={"ID":"cee4188b-7e72-4b33-bde9-d1b2a585d0b5","Type":"ContainerStarted","Data":"f22a3c21c489389cad121799986dd62991e26436c5840137e2448cb9ccad20f0"}
Sep 09 12:12:05.151666 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:05.151237    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:12:05.168881 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:05.168829    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n8bz5" podStartSLOduration=65.546832237 podStartE2EDuration="1m9.168813997s" totalImagesPullingTime="3.62198176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:05.168084058 +0000 UTC m=+69.809703263" watchObservedRunningTime="2026-09-09 12:12:05.168813997 +0000 UTC m=+69.810433198"
Sep 09 12:12:06.131141 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:06.131097    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:06.131331 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:06.131269    2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:12:06.131408 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:06.131360    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls podName:ea9590d0-7506-42a0-84e8-e498a4df5b3d nodeName:}" failed. No retries permitted until 2026-09-09 12:12:10.131338784 +0000 UTC m=+74.772957976 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-l5kgg" (UID: "ea9590d0-7506-42a0-84e8-e498a4df5b3d") : secret "samples-operator-tls" not found
Sep 09 12:12:06.433664 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:06.433576    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:06.433664 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:06.433652    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:06.434191 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:06.433664    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:12:06.434191 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:06.433746    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:14.433728018 +0000 UTC m=+79.075347201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : configmap "kube-rbac-proxy" not found
Sep 09 12:12:06.434191 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:06.433814    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:12:06.434191 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:06.433891    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:14.433875106 +0000 UTC m=+79.075494302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:12:08.160382 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:08.160348    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-m6ssq" event={"ID":"a18b39f3-278e-4cc9-97b0-867a633220f3","Type":"ContainerStarted","Data":"2fabe08f1b5c9e6c8e55773b563639b7aac661dc09acc00b8e322cb36d5205cd"}
Sep 09 12:12:08.178152 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:08.178104    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-m6ssq" podStartSLOduration=65.975388928 podStartE2EDuration="1m12.178087571s" totalImagesPullingTime="6.202698643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:08.177273385 +0000 UTC m=+72.818892589" watchObservedRunningTime="2026-09-09 12:12:08.178087571 +0000 UTC m=+72.819706808"
Sep 09 12:12:09.377614 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.377579    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-xc5gl"]
Sep 09 12:12:09.379799 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.379783    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.382066 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.382043    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 09 12:12:09.382066 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.382060    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hr8cw\""
Sep 09 12:12:09.382221 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.382046    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 09 12:12:09.382953 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.382939    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 09 12:12:09.383041 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.383025    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 09 12:12:09.388560 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.388539    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-xc5gl"]
Sep 09 12:12:09.455693 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.455657    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b741c03e-4300-4c00-b412-0e26a6c5fe13-signing-key\") pod \"service-ca-75d99c484-xc5gl\" (UID: \"b741c03e-4300-4c00-b412-0e26a6c5fe13\") " pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.455923 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.455728    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b741c03e-4300-4c00-b412-0e26a6c5fe13-signing-cabundle\") pod \"service-ca-75d99c484-xc5gl\" (UID: \"b741c03e-4300-4c00-b412-0e26a6c5fe13\") " pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.455923 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.455780    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qzcw\" (UniqueName: \"kubernetes.io/projected/b741c03e-4300-4c00-b412-0e26a6c5fe13-kube-api-access-4qzcw\") pod \"service-ca-75d99c484-xc5gl\" (UID: \"b741c03e-4300-4c00-b412-0e26a6c5fe13\") " pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.556903 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.556846    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b741c03e-4300-4c00-b412-0e26a6c5fe13-signing-key\") pod \"service-ca-75d99c484-xc5gl\" (UID: \"b741c03e-4300-4c00-b412-0e26a6c5fe13\") " pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.557070 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.556916    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b741c03e-4300-4c00-b412-0e26a6c5fe13-signing-cabundle\") pod \"service-ca-75d99c484-xc5gl\" (UID: \"b741c03e-4300-4c00-b412-0e26a6c5fe13\") " pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.557070 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.556941    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qzcw\" (UniqueName: \"kubernetes.io/projected/b741c03e-4300-4c00-b412-0e26a6c5fe13-kube-api-access-4qzcw\") pod \"service-ca-75d99c484-xc5gl\" (UID: \"b741c03e-4300-4c00-b412-0e26a6c5fe13\") " pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.557535 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.557513    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b741c03e-4300-4c00-b412-0e26a6c5fe13-signing-cabundle\") pod \"service-ca-75d99c484-xc5gl\" (UID: \"b741c03e-4300-4c00-b412-0e26a6c5fe13\") " pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.559350 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.559331    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b741c03e-4300-4c00-b412-0e26a6c5fe13-signing-key\") pod \"service-ca-75d99c484-xc5gl\" (UID: \"b741c03e-4300-4c00-b412-0e26a6c5fe13\") " pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.571987 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.571965    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qzcw\" (UniqueName: \"kubernetes.io/projected/b741c03e-4300-4c00-b412-0e26a6c5fe13-kube-api-access-4qzcw\") pod \"service-ca-75d99c484-xc5gl\" (UID: \"b741c03e-4300-4c00-b412-0e26a6c5fe13\") " pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.688480 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.688399    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-xc5gl"
Sep 09 12:12:09.955688 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:09.955614    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-xc5gl"]
Sep 09 12:12:09.958881 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:09.958840    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb741c03e_4300_4c00_b412_0e26a6c5fe13.slice/crio-a478c360182b69329ba830905c8a7da35ff7b402c845fea10a458e283df8b3e9 WatchSource:0}: Error finding container a478c360182b69329ba830905c8a7da35ff7b402c845fea10a458e283df8b3e9: Status 404 returned error can't find the container with id a478c360182b69329ba830905c8a7da35ff7b402c845fea10a458e283df8b3e9
Sep 09 12:12:10.159951 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:10.159917    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:10.160118 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:10.160027    2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:12:10.160118 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:10.160082    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls podName:ea9590d0-7506-42a0-84e8-e498a4df5b3d nodeName:}" failed. No retries permitted until 2026-09-09 12:12:18.160068009 +0000 UTC m=+82.801687191 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-l5kgg" (UID: "ea9590d0-7506-42a0-84e8-e498a4df5b3d") : secret "samples-operator-tls" not found
Sep 09 12:12:10.165930 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:10.165903    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-xc5gl" event={"ID":"b741c03e-4300-4c00-b412-0e26a6c5fe13","Type":"ContainerStarted","Data":"a478c360182b69329ba830905c8a7da35ff7b402c845fea10a458e283df8b3e9"}
Sep 09 12:12:13.175684 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:13.175645    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-xc5gl" event={"ID":"b741c03e-4300-4c00-b412-0e26a6c5fe13","Type":"ContainerStarted","Data":"0b350f0238f1842a998ff807e5d5c45b6ac6ab86f950caf5fe8a499d65623640"}
Sep 09 12:12:14.499300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:14.499262    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:14.499738 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:14.499326    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:14.499738 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:14.499439    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:12:14.499738 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:12:14.499516    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls podName:095003f4-af63-4eda-82af-671ce6a44404 nodeName:}" failed. No retries permitted until 2026-09-09 12:12:30.499496346 +0000 UTC m=+95.141115530 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8vfpn" (UID: "095003f4-af63-4eda-82af-671ce6a44404") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:12:14.499862 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:14.499789    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095003f4-af63-4eda-82af-671ce6a44404-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:18.226960 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:18.226839    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:18.229308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:18.229287    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea9590d0-7506-42a0-84e8-e498a4df5b3d-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-l5kgg\" (UID: \"ea9590d0-7506-42a0-84e8-e498a4df5b3d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:18.256791 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:18.256757    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"
Sep 09 12:12:18.528191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:18.528137    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-xc5gl" podStartSLOduration=7.2781964630000004 podStartE2EDuration="9.528109454s" totalImagesPullingTime="2.249912991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:13.192758736 +0000 UTC m=+77.834377943" watchObservedRunningTime="2026-09-09 12:12:18.528109454 +0000 UTC m=+83.169728659"
Sep 09 12:12:18.528967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:18.528946    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg"]
Sep 09 12:12:19.191691 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:19.191654    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg" event={"ID":"ea9590d0-7506-42a0-84e8-e498a4df5b3d","Type":"ContainerStarted","Data":"1fccbcf2af0a0d060a534f60a9054759c9b670fac958390ff5596f42f552d5b4"}
Sep 09 12:12:21.199054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:21.199013    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg" event={"ID":"ea9590d0-7506-42a0-84e8-e498a4df5b3d","Type":"ContainerStarted","Data":"3fcb5a4242445f792b2fae6a7c511931d5efa72168f678f1f3e370caaa8f6e51"}
Sep 09 12:12:21.199566 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:21.199069    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg" event={"ID":"ea9590d0-7506-42a0-84e8-e498a4df5b3d","Type":"ContainerStarted","Data":"f39ce66cdebacca761fe593caa6968056bb50edec339a9a2f899d5a4acbb5c6c"}
Sep 09 12:12:21.218215 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:21.218165    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-l5kgg" podStartSLOduration=17.287035644 podStartE2EDuration="19.218151705s" totalImagesPullingTime="1.931116061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:21.216815937 +0000 UTC m=+85.858435142" watchObservedRunningTime="2026-09-09 12:12:21.218151705 +0000 UTC m=+85.859770909"
Sep 09 12:12:30.262094 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.262058    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"]
Sep 09 12:12:30.265541 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.265521    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"
Sep 09 12:12:30.269363 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.269341    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 12:12:30.269480 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.269453    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-qlcl8\""
Sep 09 12:12:30.270588 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.270568    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 09 12:12:30.270711 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.270626    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 12:12:30.270711 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.270626    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 12:12:30.301707 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.301658    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"]
Sep 09 12:12:30.315796 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.315766    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b087837-7bc7-46fd-a638-bccfc3b7c611-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65df955bdd-94czv\" (UID: \"8b087837-7bc7-46fd-a638-bccfc3b7c611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"
Sep 09 12:12:30.315939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.315807    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9gcb\" (UniqueName: \"kubernetes.io/projected/8b087837-7bc7-46fd-a638-bccfc3b7c611-kube-api-access-g9gcb\") pod \"managed-serviceaccount-addon-agent-65df955bdd-94czv\" (UID: \"8b087837-7bc7-46fd-a638-bccfc3b7c611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"
Sep 09 12:12:30.348288 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.348258    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-pc62x"]
Sep 09 12:12:30.351110 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.351095    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x"
Sep 09 12:12:30.354236 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.354211    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr"]
Sep 09 12:12:30.355092 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.355070    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 09 12:12:30.355296 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.355108    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2r6kc\""
Sep 09 12:12:30.355544 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.355526    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 09 12:12:30.357104 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.357086    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr"
Sep 09 12:12:30.359235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.359217    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-b7466\""
Sep 09 12:12:30.359347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.359332    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 09 12:12:30.363156 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.363136    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-pc62x"]
Sep 09 12:12:30.373053 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.373033    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr"]
Sep 09 12:12:30.416896 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.416871    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/751eecc0-b1e9-46bc-85c2-b18006860c4c-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pc62x\" (UID: \"751eecc0-b1e9-46bc-85c2-b18006860c4c\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x"
Sep 09 12:12:30.416896 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.416902    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b087837-7bc7-46fd-a638-bccfc3b7c611-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65df955bdd-94czv\" (UID: \"8b087837-7bc7-46fd-a638-bccfc3b7c611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"
Sep 09 12:12:30.417094 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.416924    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9gcb\" (UniqueName: \"kubernetes.io/projected/8b087837-7bc7-46fd-a638-bccfc3b7c611-kube-api-access-g9gcb\") pod \"managed-serviceaccount-addon-agent-65df955bdd-94czv\" (UID: \"8b087837-7bc7-46fd-a638-bccfc3b7c611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"
Sep 09 12:12:30.417094 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.416942    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2c573e19-94a2-4af0-8f22-e25fe096f120-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-hbthr\" (UID: \"2c573e19-94a2-4af0-8f22-e25fe096f120\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr"
Sep 09 12:12:30.417094 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.416993    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/751eecc0-b1e9-46bc-85c2-b18006860c4c-nginx-conf\") pod \"networking-console-plugin-79977f9844-pc62x\" (UID: \"751eecc0-b1e9-46bc-85c2-b18006860c4c\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x"
Sep 09 12:12:30.419423 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.419408    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b087837-7bc7-46fd-a638-bccfc3b7c611-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65df955bdd-94czv\" (UID: \"8b087837-7bc7-46fd-a638-bccfc3b7c611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"
Sep 09 12:12:30.439577 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.439551    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9gcb\" (UniqueName: \"kubernetes.io/projected/8b087837-7bc7-46fd-a638-bccfc3b7c611-kube-api-access-g9gcb\") pod \"managed-serviceaccount-addon-agent-65df955bdd-94czv\" (UID: \"8b087837-7bc7-46fd-a638-bccfc3b7c611\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"
Sep 09 12:12:30.518020 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.517950    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/751eecc0-b1e9-46bc-85c2-b18006860c4c-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pc62x\" (UID: \"751eecc0-b1e9-46bc-85c2-b18006860c4c\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x"
Sep 09 12:12:30.518020 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.517988    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2c573e19-94a2-4af0-8f22-e25fe096f120-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-hbthr\" (UID: \"2c573e19-94a2-4af0-8f22-e25fe096f120\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr"
Sep 09 12:12:30.518020 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.518013    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:30.518264 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.518029    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/751eecc0-b1e9-46bc-85c2-b18006860c4c-nginx-conf\") pod \"networking-console-plugin-79977f9844-pc62x\" (UID: \"751eecc0-b1e9-46bc-85c2-b18006860c4c\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x"
Sep 09 12:12:30.518724 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.518688    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/751eecc0-b1e9-46bc-85c2-b18006860c4c-nginx-conf\") pod \"networking-console-plugin-79977f9844-pc62x\" (UID: \"751eecc0-b1e9-46bc-85c2-b18006860c4c\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x"
Sep 09 12:12:30.520366 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.520343    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095003f4-af63-4eda-82af-671ce6a44404-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vfpn\" (UID: \"095003f4-af63-4eda-82af-671ce6a44404\") " pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:30.520496 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.520475    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2c573e19-94a2-4af0-8f22-e25fe096f120-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-hbthr\" (UID: \"2c573e19-94a2-4af0-8f22-e25fe096f120\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr"
Sep 09 12:12:30.520553 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.520524    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/751eecc0-b1e9-46bc-85c2-b18006860c4c-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pc62x\" (UID: \"751eecc0-b1e9-46bc-85c2-b18006860c4c\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x"
Sep 09 12:12:30.581819 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.581787    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"
Sep 09 12:12:30.660889 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.660814    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x"
Sep 09 12:12:30.666908 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.666881    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr"
Sep 09 12:12:30.776851 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.776777    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8vfpn"
Sep 09 12:12:30.882599 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.882568    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv"]
Sep 09 12:12:30.887332 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:30.887308    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b087837_7bc7_46fd_a638_bccfc3b7c611.slice/crio-1293c613242c21c56706f7ba4f6eed0e1f93cc7d305c996f71753cbb97da167a WatchSource:0}: Error finding container 1293c613242c21c56706f7ba4f6eed0e1f93cc7d305c996f71753cbb97da167a: Status 404 returned error can't find the container with id 1293c613242c21c56706f7ba4f6eed0e1f93cc7d305c996f71753cbb97da167a
Sep 09 12:12:30.959445 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.959406    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr"]
Sep 09 12:12:30.961589 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:30.961546    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-pc62x"]
Sep 09 12:12:30.963618 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:30.963589    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c573e19_94a2_4af0_8f22_e25fe096f120.slice/crio-da8f3e68d670900e2451f32ca06b2b6fc1d96d677a2f73af1156f4c684202173 WatchSource:0}: Error finding container da8f3e68d670900e2451f32ca06b2b6fc1d96d677a2f73af1156f4c684202173: Status 404 returned error can't find the container with id da8f3e68d670900e2451f32ca06b2b6fc1d96d677a2f73af1156f4c684202173
Sep 09 12:12:30.964420 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:30.964392    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751eecc0_b1e9_46bc_85c2_b18006860c4c.slice/crio-d45a6f5bcea3e6424fe588d69ecb032c53def3af58dabedb98e8cf411a4c69c8 WatchSource:0}: Error finding container d45a6f5bcea3e6424fe588d69ecb032c53def3af58dabedb98e8cf411a4c69c8: Status 404 returned error can't find the container with id d45a6f5bcea3e6424fe588d69ecb032c53def3af58dabedb98e8cf411a4c69c8
Sep 09 12:12:31.056225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:31.056202    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8vfpn"]
Sep 09 12:12:31.058293 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:31.058269    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095003f4_af63_4eda_82af_671ce6a44404.slice/crio-9f48c55c79418d6ce2835c9585c8ec79bfd0f2ff0419c65789ea3e29188021e3 WatchSource:0}: Error finding container 9f48c55c79418d6ce2835c9585c8ec79bfd0f2ff0419c65789ea3e29188021e3: Status 404 returned error can't find the container with id 9f48c55c79418d6ce2835c9585c8ec79bfd0f2ff0419c65789ea3e29188021e3
Sep 09 12:12:31.231506 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:31.231473    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8vfpn" event={"ID":"095003f4-af63-4eda-82af-671ce6a44404","Type":"ContainerStarted","Data":"6e03d4d0be29ab9c42fbe4f193b5473f3dc9d4927760e2cb932844a1dadaee99"}
Sep 09 12:12:31.231679 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:31.231528    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8vfpn" event={"ID":"095003f4-af63-4eda-82af-671ce6a44404","Type":"ContainerStarted","Data":"9f48c55c79418d6ce2835c9585c8ec79bfd0f2ff0419c65789ea3e29188021e3"}
Sep 09 12:12:31.232676 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:31.232652    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x" event={"ID":"751eecc0-b1e9-46bc-85c2-b18006860c4c","Type":"ContainerStarted","Data":"d45a6f5bcea3e6424fe588d69ecb032c53def3af58dabedb98e8cf411a4c69c8"}
Sep 09 12:12:31.233649 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:31.233629    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr" event={"ID":"2c573e19-94a2-4af0-8f22-e25fe096f120","Type":"ContainerStarted","Data":"da8f3e68d670900e2451f32ca06b2b6fc1d96d677a2f73af1156f4c684202173"}
Sep 09 12:12:31.234661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:31.234644    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv" event={"ID":"8b087837-7bc7-46fd-a638-bccfc3b7c611","Type":"ContainerStarted","Data":"1293c613242c21c56706f7ba4f6eed0e1f93cc7d305c996f71753cbb97da167a"}
Sep 09 12:12:33.141069 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.140974    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:12:33.143994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.143962    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"image-registry-785748674c-9n6ft\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") " pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:12:33.155169 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.155145    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bkd5d\""
Sep 09 12:12:33.163136 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.163111    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:12:33.241416 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.241380    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:12:33.242921 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.242891    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8vfpn" event={"ID":"095003f4-af63-4eda-82af-671ce6a44404","Type":"ContainerStarted","Data":"6e8da4fe942592d4d6ea09b3ec4b6f9b9bd3540e280f77fd0673ac0b93527ac3"}
Sep 09 12:12:33.244259 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.244231    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46998610-68a0-4431-bc25-313e17abd0ad-metrics-tls\") pod \"dns-default-vrrhx\" (UID: \"46998610-68a0-4431-bc25-313e17abd0ad\") " pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:12:33.245109 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.244721    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x" event={"ID":"751eecc0-b1e9-46bc-85c2-b18006860c4c","Type":"ContainerStarted","Data":"a2ee80461c6b3dedff6fcaa8af1694f88d39b8fbc974f0461efaf17492906046"}
Sep 09 12:12:33.263266 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.263196    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-pc62x" podStartSLOduration=2.033289832 podStartE2EDuration="3.263179543s" totalImagesPullingTime="1.229889711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:33.262243102 +0000 UTC m=+97.903862307" watchObservedRunningTime="2026-09-09 12:12:33.263179543 +0000 UTC m=+97.904798741"
Sep 09 12:12:33.299747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.299716    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mksrl\""
Sep 09 12:12:33.309091 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.307770    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:12:33.639349 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.639317    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vrrhx"]
Sep 09 12:12:33.642848 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:33.642746    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-785748674c-9n6ft"]
Sep 09 12:12:33.643175 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:33.643143    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46998610_68a0_4431_bc25_313e17abd0ad.slice/crio-bd61580708d554f6cf0551d3328368c87a21adba41edaedfc3a0a968ce2949ae WatchSource:0}: Error finding container bd61580708d554f6cf0551d3328368c87a21adba41edaedfc3a0a968ce2949ae: Status 404 returned error can't find the container with id bd61580708d554f6cf0551d3328368c87a21adba41edaedfc3a0a968ce2949ae
Sep 09 12:12:33.645661 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:33.645637    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6955fe65_c931_4735_8315_03f05ae37092.slice/crio-a9ce223fce999376b97a0792172f22c51ec29f0b4fded80ad6217ae7043b55e0 WatchSource:0}: Error finding container a9ce223fce999376b97a0792172f22c51ec29f0b4fded80ad6217ae7043b55e0: Status 404 returned error can't find the container with id a9ce223fce999376b97a0792172f22c51ec29f0b4fded80ad6217ae7043b55e0
Sep 09 12:12:34.250815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:34.250772    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv" event={"ID":"8b087837-7bc7-46fd-a638-bccfc3b7c611","Type":"ContainerStarted","Data":"ae15dbe21f20f4383c2a954feb2e10543480425fb23e4440895161bf5c865309"}
Sep 09 12:12:34.252091 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:34.252067    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vrrhx" event={"ID":"46998610-68a0-4431-bc25-313e17abd0ad","Type":"ContainerStarted","Data":"bd61580708d554f6cf0551d3328368c87a21adba41edaedfc3a0a968ce2949ae"}
Sep 09 12:12:34.253783 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:34.253754    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-785748674c-9n6ft" event={"ID":"6955fe65-c931-4735-8315-03f05ae37092","Type":"ContainerStarted","Data":"d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7"}
Sep 09 12:12:34.253903 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:34.253791    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-785748674c-9n6ft" event={"ID":"6955fe65-c931-4735-8315-03f05ae37092","Type":"ContainerStarted","Data":"a9ce223fce999376b97a0792172f22c51ec29f0b4fded80ad6217ae7043b55e0"}
Sep 09 12:12:34.253903 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:34.253866    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:12:34.291608 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:34.291516    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65df955bdd-94czv" podStartSLOduration=1.751802268 podStartE2EDuration="4.291495137s" totalImagesPullingTime="2.539692869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:34.270260299 +0000 UTC m=+98.911879503" watchObservedRunningTime="2026-09-09 12:12:34.291495137 +0000 UTC m=+98.933114338"
Sep 09 12:12:34.292459 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:34.292418    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-785748674c-9n6ft" podStartSLOduration=98.29240397 podStartE2EDuration="1m38.29240397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:34.291105221 +0000 UTC m=+98.932724462" watchObservedRunningTime="2026-09-09 12:12:34.29240397 +0000 UTC m=+98.934023181"
Sep 09 12:12:35.259004 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:35.258959    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr" event={"ID":"2c573e19-94a2-4af0-8f22-e25fe096f120","Type":"ContainerStarted","Data":"9e8cdfae7b71e231343e5545e9592ea84042aab92833e89b113068de43049e4b"}
Sep 09 12:12:35.259459 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:35.259366    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr"
Sep 09 12:12:35.261633 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:35.261574    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8vfpn" event={"ID":"095003f4-af63-4eda-82af-671ce6a44404","Type":"ContainerStarted","Data":"75dbc99bdf9944687136ef0dd8b12e2286cf745042691d36eb78f18ce0cbeb2b"}
Sep 09 12:12:35.265166 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:35.265136    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr"
Sep 09 12:12:35.276655 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:35.276598    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-hbthr" podStartSLOduration=1.694417283 podStartE2EDuration="5.276579093s" totalImagesPullingTime="3.58216181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:35.275671958 +0000 UTC m=+99.917291163" watchObservedRunningTime="2026-09-09 12:12:35.276579093 +0000 UTC m=+99.918198300"
Sep 09 12:12:35.295805 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:35.295749    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8vfpn" podStartSLOduration=33.935197877 podStartE2EDuration="37.295727278s" totalImagesPullingTime="3.360529401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:11:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:35.29518705 +0000 UTC m=+99.936806254" watchObservedRunningTime="2026-09-09 12:12:35.295727278 +0000 UTC m=+99.937346476"
Sep 09 12:12:36.156560 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:36.156532    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n8bz5"
Sep 09 12:12:36.267978 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:36.267903    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vrrhx" event={"ID":"46998610-68a0-4431-bc25-313e17abd0ad","Type":"ContainerStarted","Data":"d15d693c3f8b6440f79e3255bf5ca9768d837c86b57e51d698a01075aa188b28"}
Sep 09 12:12:37.273142 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.273105    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vrrhx" event={"ID":"46998610-68a0-4431-bc25-313e17abd0ad","Type":"ContainerStarted","Data":"e81af817da13ae82bd89f176e74e4d4e1158791053574294a437d41878d71a56"}
Sep 09 12:12:37.291924 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.291873    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vrrhx" podStartSLOduration=65.87684301 podStartE2EDuration="1m8.291855538s" totalImagesPullingTime="2.415012528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:37.291466192 +0000 UTC m=+101.933085396" watchObservedRunningTime="2026-09-09 12:12:37.291855538 +0000 UTC m=+101.933474742"
Sep 09 12:12:37.597001 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.596931    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79d6dfdbb4-b8dqn"]
Sep 09 12:12:37.619716 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.619680    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79d6dfdbb4-b8dqn"]
Sep 09 12:12:37.619899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.619801    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.622792 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.622770    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 12:12:37.622980 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.622964    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 12:12:37.623062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.623012    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 12:12:37.623623 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.623607    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7pm6s\""
Sep 09 12:12:37.623734 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.623642    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 12:12:37.623734 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.623723    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 09 12:12:37.623950 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.623920    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 12:12:37.623950 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.623943    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 09 12:12:37.778980 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.778942    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-oauth-config\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.778980 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.778979    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2q57\" (UniqueName: \"kubernetes.io/projected/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-kube-api-access-m2q57\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.779208 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.779001    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-service-ca\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.779208 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.779031    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-config\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.779208 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.779117    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-oauth-serving-cert\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.779208 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.779150    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-serving-cert\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.880462 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.880381    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-config\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.880462 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.880436    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-oauth-serving-cert\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.880628 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.880547    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-serving-cert\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.880628 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.880602    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-oauth-config\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.880628 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.880626    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2q57\" (UniqueName: \"kubernetes.io/projected/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-kube-api-access-m2q57\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.880764 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.880647    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-service-ca\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.881093 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.881071    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-oauth-serving-cert\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.881220 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.881199    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-config\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.881283 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.881266    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-service-ca\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.883167 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.883152    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-serving-cert\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.883276 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.883257    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-oauth-config\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.890375 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.890350    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2q57\" (UniqueName: \"kubernetes.io/projected/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-kube-api-access-m2q57\") pod \"console-79d6dfdbb4-b8dqn\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") " pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:37.928737 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:37.928687    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:38.245606 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:38.245525    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79d6dfdbb4-b8dqn"]
Sep 09 12:12:38.248569 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:38.248534    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f743664_ab1a_49f3_ba0d_d2d3d76ba3b8.slice/crio-0a4554bb9a8c2e7b5648f421be9847e5542f01e3271622f67a4318d8db62f4ea WatchSource:0}: Error finding container 0a4554bb9a8c2e7b5648f421be9847e5542f01e3271622f67a4318d8db62f4ea: Status 404 returned error can't find the container with id 0a4554bb9a8c2e7b5648f421be9847e5542f01e3271622f67a4318d8db62f4ea
Sep 09 12:12:38.277774 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:38.277732    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d6dfdbb4-b8dqn" event={"ID":"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8","Type":"ContainerStarted","Data":"0a4554bb9a8c2e7b5648f421be9847e5542f01e3271622f67a4318d8db62f4ea"}
Sep 09 12:12:38.278098 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:38.277982    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:12:41.289997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:41.289959    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d6dfdbb4-b8dqn" event={"ID":"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8","Type":"ContainerStarted","Data":"f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c"}
Sep 09 12:12:41.312307 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:41.312255    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79d6dfdbb4-b8dqn" podStartSLOduration=1.41753608 podStartE2EDuration="4.312238206s" totalImagesPullingTime="2.894702126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:41.311500613 +0000 UTC m=+105.953119839" watchObservedRunningTime="2026-09-09 12:12:41.312238206 +0000 UTC m=+105.953857410"
Sep 09 12:12:43.107836 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.107798    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"]
Sep 09 12:12:43.111545 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.111526    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.114527 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.114369    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 12:12:43.114527 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.114371    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 09 12:12:43.114527 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.114374    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2m696\""
Sep 09 12:12:43.114527 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.114423    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 09 12:12:43.115508 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.115492    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 12:12:43.116260 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.116239    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 12:12:43.123622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.123601    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"]
Sep 09 12:12:43.126203 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.126180    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"]
Sep 09 12:12:43.138485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.138450    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.141312 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.141170    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 09 12:12:43.141929 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.141556    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 09 12:12:43.141929 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.141788    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fc75p\""
Sep 09 12:12:43.146077 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.146057    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"]
Sep 09 12:12:43.148012 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.147995    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wgc84"]
Sep 09 12:12:43.148722 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.148684    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 09 12:12:43.153076 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.153060    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.156000 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.155981    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 12:12:43.156096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.156019    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xwtql\""
Sep 09 12:12:43.156194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.156170    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 12:12:43.156285 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.156222    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 12:12:43.224557 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.224526    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/231e8ac2-4546-4653-923a-4da453187e31-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.224557 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.224559    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/231e8ac2-4546-4653-923a-4da453187e31-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.224821 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.224579    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231e8ac2-4546-4653-923a-4da453187e31-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.224821 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.224605    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdzjr\" (UniqueName: \"kubernetes.io/projected/af02d5ca-a5b8-4588-9d21-e43935461318-kube-api-access-gdzjr\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.224821 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.224728    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af02d5ca-a5b8-4588-9d21-e43935461318-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.224821 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.224764    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af02d5ca-a5b8-4588-9d21-e43935461318-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.225023 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.224827    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/231e8ac2-4546-4653-923a-4da453187e31-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.225023 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.224854    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af02d5ca-a5b8-4588-9d21-e43935461318-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.225023 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.224880    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/231e8ac2-4546-4653-923a-4da453187e31-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.225023 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.224903    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mghbn\" (UniqueName: \"kubernetes.io/projected/231e8ac2-4546-4653-923a-4da453187e31-kube-api-access-mghbn\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.326233 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326190    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-wtmp\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.326427 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326247    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/231e8ac2-4546-4653-923a-4da453187e31-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.326427 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326313    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/231e8ac2-4546-4653-923a-4da453187e31-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.326427 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326350    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231e8ac2-4546-4653-923a-4da453187e31-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.326427 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326389    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdzjr\" (UniqueName: \"kubernetes.io/projected/af02d5ca-a5b8-4588-9d21-e43935461318-kube-api-access-gdzjr\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.326427 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326418    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af02d5ca-a5b8-4588-9d21-e43935461318-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.326671 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326445    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-textfile\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.326671 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326472    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af02d5ca-a5b8-4588-9d21-e43935461318-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.326671 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326498    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g54f7\" (UniqueName: \"kubernetes.io/projected/f85167fb-19a2-49b2-83fe-bdc865773943-kube-api-access-g54f7\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.326844 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326745    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f85167fb-19a2-49b2-83fe-bdc865773943-sys\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.326844 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326793    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f85167fb-19a2-49b2-83fe-bdc865773943-root\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.326844 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326822    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-tls\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.326999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326874    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/231e8ac2-4546-4653-923a-4da453187e31-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.326999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326900    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af02d5ca-a5b8-4588-9d21-e43935461318-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.326999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326923    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.326999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326954    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/231e8ac2-4546-4653-923a-4da453187e31-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.326999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.326976    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mghbn\" (UniqueName: \"kubernetes.io/projected/231e8ac2-4546-4653-923a-4da453187e31-kube-api-access-mghbn\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.327235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.327032    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f85167fb-19a2-49b2-83fe-bdc865773943-metrics-client-ca\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.327235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.327042    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/231e8ac2-4546-4653-923a-4da453187e31-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.327235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.327059    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-accelerators-collector-config\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.327235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.327081    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231e8ac2-4546-4653-923a-4da453187e31-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.327428 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.327229    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/231e8ac2-4546-4653-923a-4da453187e31-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.328012 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.327968    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af02d5ca-a5b8-4588-9d21-e43935461318-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.329372 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.329339    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af02d5ca-a5b8-4588-9d21-e43935461318-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.329496 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.329340    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/231e8ac2-4546-4653-923a-4da453187e31-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.329663 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.329643    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af02d5ca-a5b8-4588-9d21-e43935461318-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.329852 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.329833    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/231e8ac2-4546-4653-923a-4da453187e31-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.339310 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.339277    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mghbn\" (UniqueName: \"kubernetes.io/projected/231e8ac2-4546-4653-923a-4da453187e31-kube-api-access-mghbn\") pod \"kube-state-metrics-74cb57df7-tlvc6\" (UID: \"231e8ac2-4546-4653-923a-4da453187e31\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.339421 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.339381    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdzjr\" (UniqueName: \"kubernetes.io/projected/af02d5ca-a5b8-4588-9d21-e43935461318-kube-api-access-gdzjr\") pod \"openshift-state-metrics-657fd6c54-br2tn\" (UID: \"af02d5ca-a5b8-4588-9d21-e43935461318\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.423686 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.423610    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"
Sep 09 12:12:43.427665 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.427632    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-textfile\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.427820 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.427675    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g54f7\" (UniqueName: \"kubernetes.io/projected/f85167fb-19a2-49b2-83fe-bdc865773943-kube-api-access-g54f7\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.427820 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.427714    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f85167fb-19a2-49b2-83fe-bdc865773943-sys\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.427820 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.427746    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f85167fb-19a2-49b2-83fe-bdc865773943-root\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.427820 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.427770    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-tls\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.428418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.427829    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f85167fb-19a2-49b2-83fe-bdc865773943-sys\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.428418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.427845    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f85167fb-19a2-49b2-83fe-bdc865773943-root\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.428418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.427897    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.428418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.427965    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f85167fb-19a2-49b2-83fe-bdc865773943-metrics-client-ca\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.428418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.427992    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-accelerators-collector-config\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.428418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.428101    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-wtmp\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.428418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.428282    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-wtmp\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.428418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.428039    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-textfile\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.429056 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.428538    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-accelerators-collector-config\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.429056 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.428789    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f85167fb-19a2-49b2-83fe-bdc865773943-metrics-client-ca\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.430317 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.430296    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-tls\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.430572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.430548    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f85167fb-19a2-49b2-83fe-bdc865773943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.437625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.437595    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g54f7\" (UniqueName: \"kubernetes.io/projected/f85167fb-19a2-49b2-83fe-bdc865773943-kube-api-access-g54f7\") pod \"node-exporter-wgc84\" (UID: \"f85167fb-19a2-49b2-83fe-bdc865773943\") " pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.455334 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.455302    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"
Sep 09 12:12:43.465453 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.465432    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wgc84"
Sep 09 12:12:43.478580 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:43.478529    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85167fb_19a2_49b2_83fe_bdc865773943.slice/crio-5e0fb4fd98d22b2ab6637b56623356e697431137f49ce12b8369a4adff2aa9bf WatchSource:0}: Error finding container 5e0fb4fd98d22b2ab6637b56623356e697431137f49ce12b8369a4adff2aa9bf: Status 404 returned error can't find the container with id 5e0fb4fd98d22b2ab6637b56623356e697431137f49ce12b8369a4adff2aa9bf
Sep 09 12:12:43.699733 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.699690    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn"]
Sep 09 12:12:43.701786 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:43.701759    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf02d5ca_a5b8_4588_9d21_e43935461318.slice/crio-2b020fa975b4532fe4f09512b985ec2c7c92bce544dd7a6f489e6e6a5308ff88 WatchSource:0}: Error finding container 2b020fa975b4532fe4f09512b985ec2c7c92bce544dd7a6f489e6e6a5308ff88: Status 404 returned error can't find the container with id 2b020fa975b4532fe4f09512b985ec2c7c92bce544dd7a6f489e6e6a5308ff88
Sep 09 12:12:43.738330 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:43.738304    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6"]
Sep 09 12:12:43.755657 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:43.755621    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod231e8ac2_4546_4653_923a_4da453187e31.slice/crio-8fc74dc920b1ae82d040f65a3c5eb09640ca29bfccd42be365295a76b0bdda1c WatchSource:0}: Error finding container 8fc74dc920b1ae82d040f65a3c5eb09640ca29bfccd42be365295a76b0bdda1c: Status 404 returned error can't find the container with id 8fc74dc920b1ae82d040f65a3c5eb09640ca29bfccd42be365295a76b0bdda1c
Sep 09 12:12:44.202626 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.202597    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:12:44.206503 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.206471    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.210661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.210619    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 09 12:12:44.210805 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.210661    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ncnd6\""
Sep 09 12:12:44.211485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.210910    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 09 12:12:44.211485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.210952    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 09 12:12:44.211485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.211008    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 09 12:12:44.211485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.211102    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 09 12:12:44.211485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.210959    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 09 12:12:44.211485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.211217    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 09 12:12:44.211485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.211292    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 09 12:12:44.211485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.211326    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 09 12:12:44.224106 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.224084    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:12:44.303437 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.303400    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgc84" event={"ID":"f85167fb-19a2-49b2-83fe-bdc865773943","Type":"ContainerStarted","Data":"5e0fb4fd98d22b2ab6637b56623356e697431137f49ce12b8369a4adff2aa9bf"}
Sep 09 12:12:44.305064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.304997    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6" event={"ID":"231e8ac2-4546-4653-923a-4da453187e31","Type":"ContainerStarted","Data":"8fc74dc920b1ae82d040f65a3c5eb09640ca29bfccd42be365295a76b0bdda1c"}
Sep 09 12:12:44.307616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.307595    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn" event={"ID":"af02d5ca-a5b8-4588-9d21-e43935461318","Type":"ContainerStarted","Data":"695a49634981fba719b06be04df2d7875b0369246c363686679e8cf115aad312"}
Sep 09 12:12:44.307720 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.307646    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn" event={"ID":"af02d5ca-a5b8-4588-9d21-e43935461318","Type":"ContainerStarted","Data":"4848de21c7332a57a4d254f37a7ffe8747eac41f504efab212e8b65e9cee0f8d"}
Sep 09 12:12:44.307720 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.307676    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn" event={"ID":"af02d5ca-a5b8-4588-9d21-e43935461318","Type":"ContainerStarted","Data":"2b020fa975b4532fe4f09512b985ec2c7c92bce544dd7a6f489e6e6a5308ff88"}
Sep 09 12:12:44.335941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.335912    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.335941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.335946    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336116 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336014    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-config-out\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336116 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336042    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336116 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336085    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-web-config\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336116 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336110    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336137    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336157    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336176    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-config-volume\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336201    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336224    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs96m\" (UniqueName: \"kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-kube-api-access-zs96m\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336249    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.336308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.336280    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.436918 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.436875    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437103 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.436929    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437103 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.436977    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437103 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.437039    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437103 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.437095    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-config-out\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.437124    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.437185    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-web-config\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.437214    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.437245    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.437280    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.437298    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-config-volume\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.437316    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.437572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.437334    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs96m\" (UniqueName: \"kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-kube-api-access-zs96m\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.439223 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.439191    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.439525 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.439499    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.441203 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.440816    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.441203 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.441104    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-web-config\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.441203 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.441160    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.443126 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.443098    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.443318 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.443287    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.443801 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.443693    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.443897 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.443817    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.443980 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.443959    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.444359 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.444319    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-config-volume\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.445798 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.445776    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-config-out\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.447224 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.447205    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs96m\" (UniqueName: \"kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-kube-api-access-zs96m\") pod \"alertmanager-main-0\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.521019 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.520984    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:12:44.838323 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:44.838242    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:12:45.211059 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:45.210981    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b16436_49e3_451f_8559_51f86adf2fb9.slice/crio-298fd9a3a9719a43ca407bd91687adc4aa5bba9dfe4282fbf18a239bb0a1c10c WatchSource:0}: Error finding container 298fd9a3a9719a43ca407bd91687adc4aa5bba9dfe4282fbf18a239bb0a1c10c: Status 404 returned error can't find the container with id 298fd9a3a9719a43ca407bd91687adc4aa5bba9dfe4282fbf18a239bb0a1c10c
Sep 09 12:12:45.312963 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:45.312933    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerStarted","Data":"298fd9a3a9719a43ca407bd91687adc4aa5bba9dfe4282fbf18a239bb0a1c10c"}
Sep 09 12:12:45.314763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:45.314734    2585 generic.go:356] "Generic (PLEG): container finished" podID="f85167fb-19a2-49b2-83fe-bdc865773943" containerID="1a501e03011439cde7a7a2a39d4ecf4c3d38481bef87a1f1bc36d50d2f0cec13" exitCode=0
Sep 09 12:12:45.314837 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:45.314814    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgc84" event={"ID":"f85167fb-19a2-49b2-83fe-bdc865773943","Type":"ContainerDied","Data":"1a501e03011439cde7a7a2a39d4ecf4c3d38481bef87a1f1bc36d50d2f0cec13"}
Sep 09 12:12:46.321134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.321103    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn" event={"ID":"af02d5ca-a5b8-4588-9d21-e43935461318","Type":"ContainerStarted","Data":"f7527cd03336708c4559d3db083b1238544fe626b5cd87c4f58becb86f5a7e41"}
Sep 09 12:12:46.322610 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.322585    2585 generic.go:356] "Generic (PLEG): container finished" podID="63b16436-49e3-451f-8559-51f86adf2fb9" containerID="f947608ef629ca3e38ab5539c88e284ade9b12fff38d8c4a6bd4306809faf1b3" exitCode=0
Sep 09 12:12:46.322757 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.322668    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerDied","Data":"f947608ef629ca3e38ab5539c88e284ade9b12fff38d8c4a6bd4306809faf1b3"}
Sep 09 12:12:46.325071 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.325050    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgc84" event={"ID":"f85167fb-19a2-49b2-83fe-bdc865773943","Type":"ContainerStarted","Data":"27a670784a8dbf7fc6f62c4c6790cc630a5e8862341ca1c7b890fd6eeb85ef85"}
Sep 09 12:12:46.325178 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.325098    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgc84" event={"ID":"f85167fb-19a2-49b2-83fe-bdc865773943","Type":"ContainerStarted","Data":"b9f808b3f57752a4344e16795ca338ce9d194247e64e73424aae3d8f3b55268b"}
Sep 09 12:12:46.327391 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.327362    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6" event={"ID":"231e8ac2-4546-4653-923a-4da453187e31","Type":"ContainerStarted","Data":"900f12249a657df62f17f9a207592c7e5c6879b47596415eb48aefee4432c74e"}
Sep 09 12:12:46.327476 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.327413    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6" event={"ID":"231e8ac2-4546-4653-923a-4da453187e31","Type":"ContainerStarted","Data":"b569e149d7fddd4330a04735c9a576fd776c10b39738a80577bc731d112eb0bb"}
Sep 09 12:12:46.327476 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.327433    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6" event={"ID":"231e8ac2-4546-4653-923a-4da453187e31","Type":"ContainerStarted","Data":"a118944eaa7a030e79e1e995323f42b098c1ac503b408de642142605b5ab0bbc"}
Sep 09 12:12:46.348545 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.348492    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-br2tn" podStartSLOduration=1.918744845 podStartE2EDuration="3.348475922s" totalImagesPullingTime="1.429731077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:46.347396776 +0000 UTC m=+110.989015981" watchObservedRunningTime="2026-09-09 12:12:46.348475922 +0000 UTC m=+110.990095126"
Sep 09 12:12:46.435011 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.434960    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tlvc6" podStartSLOduration=1.930616123 podStartE2EDuration="3.434942654s" totalImagesPullingTime="1.504326531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:46.408621886 +0000 UTC m=+111.050241090" watchObservedRunningTime="2026-09-09 12:12:46.434942654 +0000 UTC m=+111.076561858"
Sep 09 12:12:46.435188 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:46.435054    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wgc84" podStartSLOduration=2.28459603 podStartE2EDuration="3.435050169s" totalImagesPullingTime="1.073035451s" totalInitContainerRuntime="77.418688ms" isStatefulPod=true podCreationTimestamp="2026-09-09 12:12:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:46.433980163 +0000 UTC m=+111.075599367" watchObservedRunningTime="2026-09-09 12:12:46.435050169 +0000 UTC m=+111.076669372"
Sep 09 12:12:47.493968 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.493935    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-b8fcf97b4-27xgf"]
Sep 09 12:12:47.497835 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.497807    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.500068 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.500031    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 09 12:12:47.500225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.500208    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-41kmf7vb64545\""
Sep 09 12:12:47.501029 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.501008    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 09 12:12:47.501029 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.501020    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 09 12:12:47.501185 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.501027    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 09 12:12:47.503556 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.503537    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qq5kw\""
Sep 09 12:12:47.508761 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.508739    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b8fcf97b4-27xgf"]
Sep 09 12:12:47.666124 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.666098    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f8d64d62-3e0b-4dc9-9244-685de948488a-metrics-server-audit-profiles\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.666249 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.666136    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f8d64d62-3e0b-4dc9-9244-685de948488a-secret-metrics-server-tls\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.666249 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.666156    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f8d64d62-3e0b-4dc9-9244-685de948488a-audit-log\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.666249 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.666211    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4rkh\" (UniqueName: \"kubernetes.io/projected/f8d64d62-3e0b-4dc9-9244-685de948488a-kube-api-access-m4rkh\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.666395 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.666266    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f8d64d62-3e0b-4dc9-9244-685de948488a-secret-metrics-server-client-certs\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.666395 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.666326    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8d64d62-3e0b-4dc9-9244-685de948488a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.666395 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.666362    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8d64d62-3e0b-4dc9-9244-685de948488a-client-ca-bundle\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.767396 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.767367    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f8d64d62-3e0b-4dc9-9244-685de948488a-metrics-server-audit-profiles\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.767541 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.767420    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f8d64d62-3e0b-4dc9-9244-685de948488a-secret-metrics-server-tls\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.767541 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.767444    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f8d64d62-3e0b-4dc9-9244-685de948488a-audit-log\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.767541 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.767475    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4rkh\" (UniqueName: \"kubernetes.io/projected/f8d64d62-3e0b-4dc9-9244-685de948488a-kube-api-access-m4rkh\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.767541 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.767516    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f8d64d62-3e0b-4dc9-9244-685de948488a-secret-metrics-server-client-certs\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.767824 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.767572    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8d64d62-3e0b-4dc9-9244-685de948488a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.767824 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.767604    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8d64d62-3e0b-4dc9-9244-685de948488a-client-ca-bundle\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.769736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.768288    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f8d64d62-3e0b-4dc9-9244-685de948488a-audit-log\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.769736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.769602    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f8d64d62-3e0b-4dc9-9244-685de948488a-metrics-server-audit-profiles\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.770475 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.770447    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8d64d62-3e0b-4dc9-9244-685de948488a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.771062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.771038    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8d64d62-3e0b-4dc9-9244-685de948488a-client-ca-bundle\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.772937 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.772915    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f8d64d62-3e0b-4dc9-9244-685de948488a-secret-metrics-server-client-certs\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.773910 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.773886    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f8d64d62-3e0b-4dc9-9244-685de948488a-secret-metrics-server-tls\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.785785 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.785762    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4rkh\" (UniqueName: \"kubernetes.io/projected/f8d64d62-3e0b-4dc9-9244-685de948488a-kube-api-access-m4rkh\") pod \"metrics-server-b8fcf97b4-27xgf\" (UID: \"f8d64d62-3e0b-4dc9-9244-685de948488a\") " pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.809766 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.809739    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:12:47.929109 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.929077    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:47.929251 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.929132    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:47.934194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:47.934011    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:48.091866 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:48.091830    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b8fcf97b4-27xgf"]
Sep 09 12:12:48.094440 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:48.094415    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d64d62_3e0b_4dc9_9244_685de948488a.slice/crio-cf271609ec080d0b8063a37747c27a49df86b8ff8c3364146d264cc89d711879 WatchSource:0}: Error finding container cf271609ec080d0b8063a37747c27a49df86b8ff8c3364146d264cc89d711879: Status 404 returned error can't find the container with id cf271609ec080d0b8063a37747c27a49df86b8ff8c3364146d264cc89d711879
Sep 09 12:12:48.285084 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:48.285055    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vrrhx"
Sep 09 12:12:48.339730 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:48.339670    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerStarted","Data":"bc60f8eba60422f13b5c433e2968783db44276c6a9bf674d18709585f30a33ed"}
Sep 09 12:12:48.339911 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:48.339781    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerStarted","Data":"47c13d3cfcc7406004634dd3bb4e00db543d0255bf7daf04c86288174187c4ce"}
Sep 09 12:12:48.339911 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:48.339826    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerStarted","Data":"3ec1670c6c45f163bab8a1f78df687d5448fc7e7bce39f680d2524c45b9e7347"}
Sep 09 12:12:48.339911 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:48.339866    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerStarted","Data":"d926857dc2005b460ca5eb1f46b487f57ab91d8c4a1542d5fe20b7356d29d75a"}
Sep 09 12:12:48.339911 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:48.339903    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerStarted","Data":"565fea6c2d81b474081a06541c7057fe64b4bd676867214b343923bea70a02fb"}
Sep 09 12:12:48.340998 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:48.340966    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf" event={"ID":"f8d64d62-3e0b-4dc9-9244-685de948488a","Type":"ContainerStarted","Data":"cf271609ec080d0b8063a37747c27a49df86b8ff8c3364146d264cc89d711879"}
Sep 09 12:12:48.345436 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:48.345333    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:12:49.349123 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.349083    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerStarted","Data":"227d809c7223ae93765f8e3e91746b27641203cd9663780eb90ff2c2e3e0e981"}
Sep 09 12:12:49.395288 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.394097    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.385547828 podStartE2EDuration="5.394076989s" totalImagesPullingTime="2.992081143s" totalInitContainerRuntime="16.448018ms" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:49.393156815 +0000 UTC m=+114.034776031" watchObservedRunningTime="2026-09-09 12:12:49.394076989 +0000 UTC m=+114.035696193"
Sep 09 12:12:49.397034 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.397010    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 12:12:49.401288 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.401265    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.404629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.404445    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 09 12:12:49.406213 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406114    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 09 12:12:49.406213 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406140    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 09 12:12:49.406213 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406121    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 09 12:12:49.406409 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406273    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-94fanvck97soo\""
Sep 09 12:12:49.406409 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406404    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 09 12:12:49.406510 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406422    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 09 12:12:49.406595 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406557    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-22fkk\""
Sep 09 12:12:49.406595 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406590    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 09 12:12:49.406850 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406743    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 09 12:12:49.406850 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406801    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 09 12:12:49.406985 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406968    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 09 12:12:49.407028 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.406982    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 09 12:12:49.407332 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.407312    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 09 12:12:49.409023 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.408960    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 09 12:12:49.414529 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.414510    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 09 12:12:49.427491 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.427404    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 12:12:49.585347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585310    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbn7w\" (UniqueName: \"kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-kube-api-access-dbn7w\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585522 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585358    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585522 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585384    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585522 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585411    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config-out\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585522 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585430    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585522 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585470    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585522 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585491    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585522 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585516    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585571    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585613    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585630    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585673    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585689    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585743    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585766    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.585815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585805    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.586187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585833    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.586187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585850    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-web-config\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.586187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585875    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.586187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.585898    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.686878 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.686790    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.686878 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.686825    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.686878 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.686845    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.686878 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.686863    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.686888    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.686915    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687007    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687042    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687075    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687104    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687139    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687183    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687211    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-web-config\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687247    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687271    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687325    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbn7w\" (UniqueName: \"kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-kube-api-access-dbn7w\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687353    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687382    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687407    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config-out\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.687547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687434    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.688101 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.687666    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.688101 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.688016    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.689102 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.688808    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.689242 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.689224    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.689941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.689917    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.690371 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.690348    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.690676 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.690656    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.690788 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.690734    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.692485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.692153    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.692485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.692263    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.692485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.692447    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.693061 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.693010    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.693232 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.693207    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.693536 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.693491    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.693536 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.693490    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.693956 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.693934    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config-out\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.694031 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.693962    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.694138 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.694115    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.694608 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.694589    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-web-config\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.697354 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.697331    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbn7w\" (UniqueName: \"kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-kube-api-access-dbn7w\") pod \"prometheus-k8s-0\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:49.714444 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:49.714421    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:12:50.002233 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:50.002166    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 12:12:50.004974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:12:50.004944    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85cb5b63_1148_422f_b4f2_b0b7c0664e1d.slice/crio-3207cfd956c418846c4cdc10964388eac195c19ca7e178f3f7f992b45368c0df WatchSource:0}: Error finding container 3207cfd956c418846c4cdc10964388eac195c19ca7e178f3f7f992b45368c0df: Status 404 returned error can't find the container with id 3207cfd956c418846c4cdc10964388eac195c19ca7e178f3f7f992b45368c0df
Sep 09 12:12:50.354107 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:50.354067    2585 generic.go:356] "Generic (PLEG): container finished" podID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerID="e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07" exitCode=0
Sep 09 12:12:50.354516 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:50.354154    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerDied","Data":"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"}
Sep 09 12:12:50.354516 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:50.354251    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerStarted","Data":"3207cfd956c418846c4cdc10964388eac195c19ca7e178f3f7f992b45368c0df"}
Sep 09 12:12:50.355766 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:50.355737    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf" event={"ID":"f8d64d62-3e0b-4dc9-9244-685de948488a","Type":"ContainerStarted","Data":"55a6b4483c845a019301cfc4b9a0c9e2b855927b96c6310d46d85945de7e33e6"}
Sep 09 12:12:50.402598 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:50.402533    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf" podStartSLOduration=2.003649264 podStartE2EDuration="3.402512428s" totalImagesPullingTime="1.398863164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:50.400944879 +0000 UTC m=+115.042564084" watchObservedRunningTime="2026-09-09 12:12:50.402512428 +0000 UTC m=+115.044131631"
Sep 09 12:12:53.802219 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:53.802121    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-785748674c-9n6ft"]
Sep 09 12:12:53.806746 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:53.806719    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:12:54.376148 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:54.376113    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerStarted","Data":"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"}
Sep 09 12:12:54.376333 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:54.376185    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerStarted","Data":"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"}
Sep 09 12:12:55.383468 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:55.383441    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerStarted","Data":"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"}
Sep 09 12:12:55.383826 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:55.383503    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerStarted","Data":"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"}
Sep 09 12:12:56.390687 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:56.390650    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerStarted","Data":"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"}
Sep 09 12:12:56.391083 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:56.390735    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerStarted","Data":"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"}
Sep 09 12:12:56.422778 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:56.422300    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.676317312 podStartE2EDuration="7.422277897s" totalImagesPullingTime="4.730022974s" totalInitContainerRuntime="15.937611ms" isStatefulPod=false podCreationTimestamp="2026-09-09 12:12:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:12:56.420133622 +0000 UTC m=+121.061752864" watchObservedRunningTime="2026-09-09 12:12:56.422277897 +0000 UTC m=+121.063897103"
Sep 09 12:12:59.715132 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:12:59.715092    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:13:03.972367 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:03.972335    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-7cdcfcf76c-l5kgg_ea9590d0-7506-42a0-84e8-e498a4df5b3d/cluster-samples-operator/0.log"
Sep 09 12:13:03.981043 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:03.981011    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-7cdcfcf76c-l5kgg_ea9590d0-7506-42a0-84e8-e498a4df5b3d/cluster-samples-operator-watch/0.log"
Sep 09 12:13:04.624611 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:04.624567    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:13:04.627060 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:04.627039    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0238389f-6661-453f-ba7b-b33a3c34271c-metrics-certs\") pod \"network-metrics-daemon-dj698\" (UID: \"0238389f-6661-453f-ba7b-b33a3c34271c\") " pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:13:04.809252 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:04.809218    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6s86q\""
Sep 09 12:13:04.817256 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:04.817229    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dj698"
Sep 09 12:13:05.105124 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:05.105088    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dj698"]
Sep 09 12:13:05.113036 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:13:05.113008    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0238389f_6661_453f_ba7b_b33a3c34271c.slice/crio-e34786d24c9182660ad024888f2d3d75c8aed4bc36de4e49f0164807b051500e WatchSource:0}: Error finding container e34786d24c9182660ad024888f2d3d75c8aed4bc36de4e49f0164807b051500e: Status 404 returned error can't find the container with id e34786d24c9182660ad024888f2d3d75c8aed4bc36de4e49f0164807b051500e
Sep 09 12:13:05.426653 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:05.426575    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dj698" event={"ID":"0238389f-6661-453f-ba7b-b33a3c34271c","Type":"ContainerStarted","Data":"e34786d24c9182660ad024888f2d3d75c8aed4bc36de4e49f0164807b051500e"}
Sep 09 12:13:05.462712 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:05.462676    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79d6dfdbb4-b8dqn"]
Sep 09 12:13:07.810771 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:07.810743    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:13:07.811185 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:07.810810    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:13:08.440266 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:08.440229    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dj698" event={"ID":"0238389f-6661-453f-ba7b-b33a3c34271c","Type":"ContainerStarted","Data":"2c7c4e3accb791d7276fcfda32a84c9fdf3feeff958f3e272a61cfc68c04059e"}
Sep 09 12:13:08.440457 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:08.440286    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dj698" event={"ID":"0238389f-6661-453f-ba7b-b33a3c34271c","Type":"ContainerStarted","Data":"c19f844c8cfebb6c65ca382b7259cb2727249cd1de2f2a358c9bcd63eec8534b"}
Sep 09 12:13:08.461597 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:08.461552    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dj698" podStartSLOduration=129.976605028 podStartE2EDuration="2m12.461536557s" totalImagesPullingTime="2.484931529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:13:08.460462236 +0000 UTC m=+133.102081439" watchObservedRunningTime="2026-09-09 12:13:08.461536557 +0000 UTC m=+133.103155760"
Sep 09 12:13:18.824555 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:18.824493    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-785748674c-9n6ft" podUID="6955fe65-c931-4735-8315-03f05ae37092" containerName="registry" containerID="cri-o://d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7" gracePeriod=30
Sep 09 12:13:19.070110 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.070085    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:13:19.152587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.152513    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vntd\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-kube-api-access-8vntd\") pod \"6955fe65-c931-4735-8315-03f05ae37092\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") "
Sep 09 12:13:19.152587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.152555    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-trusted-ca\") pod \"6955fe65-c931-4735-8315-03f05ae37092\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") "
Sep 09 12:13:19.152587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.152576    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6955fe65-c931-4735-8315-03f05ae37092-ca-trust-extracted\") pod \"6955fe65-c931-4735-8315-03f05ae37092\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") "
Sep 09 12:13:19.152886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.152607    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-registry-certificates\") pod \"6955fe65-c931-4735-8315-03f05ae37092\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") "
Sep 09 12:13:19.152886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.152645    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-bound-sa-token\") pod \"6955fe65-c931-4735-8315-03f05ae37092\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") "
Sep 09 12:13:19.152886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.152678    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-image-registry-private-configuration\") pod \"6955fe65-c931-4735-8315-03f05ae37092\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") "
Sep 09 12:13:19.152886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.152761    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") pod \"6955fe65-c931-4735-8315-03f05ae37092\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") "
Sep 09 12:13:19.152886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.152809    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-installation-pull-secrets\") pod \"6955fe65-c931-4735-8315-03f05ae37092\" (UID: \"6955fe65-c931-4735-8315-03f05ae37092\") "
Sep 09 12:13:19.153123 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.153050    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6955fe65-c931-4735-8315-03f05ae37092" (UID: "6955fe65-c931-4735-8315-03f05ae37092"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:13:19.153170 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.153114    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6955fe65-c931-4735-8315-03f05ae37092" (UID: "6955fe65-c931-4735-8315-03f05ae37092"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:13:19.155220 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.155190    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-kube-api-access-8vntd" (OuterVolumeSpecName: "kube-api-access-8vntd") pod "6955fe65-c931-4735-8315-03f05ae37092" (UID: "6955fe65-c931-4735-8315-03f05ae37092"). InnerVolumeSpecName "kube-api-access-8vntd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:13:19.155637 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.155600    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6955fe65-c931-4735-8315-03f05ae37092" (UID: "6955fe65-c931-4735-8315-03f05ae37092"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:13:19.155637 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.155615    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6955fe65-c931-4735-8315-03f05ae37092" (UID: "6955fe65-c931-4735-8315-03f05ae37092"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:13:19.155806 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.155688    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6955fe65-c931-4735-8315-03f05ae37092" (UID: "6955fe65-c931-4735-8315-03f05ae37092"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:13:19.155806 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.155729    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6955fe65-c931-4735-8315-03f05ae37092" (UID: "6955fe65-c931-4735-8315-03f05ae37092"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:13:19.163363 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.163336    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6955fe65-c931-4735-8315-03f05ae37092-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6955fe65-c931-4735-8315-03f05ae37092" (UID: "6955fe65-c931-4735-8315-03f05ae37092"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:13:19.254291 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.254258    2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-installation-pull-secrets\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:19.254291 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.254284    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vntd\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-kube-api-access-8vntd\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:19.254291 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.254294    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-trusted-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:19.254493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.254302    2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6955fe65-c931-4735-8315-03f05ae37092-ca-trust-extracted\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:19.254493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.254311    2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6955fe65-c931-4735-8315-03f05ae37092-registry-certificates\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:19.254493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.254319    2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-bound-sa-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:19.254493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.254327    2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6955fe65-c931-4735-8315-03f05ae37092-image-registry-private-configuration\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:19.254493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.254336    2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6955fe65-c931-4735-8315-03f05ae37092-registry-tls\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:19.488613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.488534    2585 generic.go:356] "Generic (PLEG): container finished" podID="6955fe65-c931-4735-8315-03f05ae37092" containerID="d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7" exitCode=0
Sep 09 12:13:19.488613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.488568    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-785748674c-9n6ft" event={"ID":"6955fe65-c931-4735-8315-03f05ae37092","Type":"ContainerDied","Data":"d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7"}
Sep 09 12:13:19.488613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.488580    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785748674c-9n6ft"
Sep 09 12:13:19.488613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.488601    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-785748674c-9n6ft" event={"ID":"6955fe65-c931-4735-8315-03f05ae37092","Type":"ContainerDied","Data":"a9ce223fce999376b97a0792172f22c51ec29f0b4fded80ad6217ae7043b55e0"}
Sep 09 12:13:19.488911 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.488625    2585 scope.go:117] "RemoveContainer" containerID="d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7"
Sep 09 12:13:19.497605 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.497589    2585 scope.go:117] "RemoveContainer" containerID="d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7"
Sep 09 12:13:19.497886 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:13:19.497867    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7\": container with ID starting with d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7 not found: ID does not exist" containerID="d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7"
Sep 09 12:13:19.497966 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.497900    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7"} err="failed to get container status \"d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7\": rpc error: code = NotFound desc = could not find container \"d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7\": container with ID starting with d775bd47044e9d68a03342a0ed97880f516c8f93216f374e8f4ded93c558d6b7 not found: ID does not exist"
Sep 09 12:13:19.510960 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.510930    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-785748674c-9n6ft"]
Sep 09 12:13:19.516103 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.516075    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-785748674c-9n6ft"]
Sep 09 12:13:19.899195 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:19.899163    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6955fe65-c931-4735-8315-03f05ae37092" path="/var/lib/kubelet/pods/6955fe65-c931-4735-8315-03f05ae37092/volumes"
Sep 09 12:13:27.815258 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:27.815226    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:13:27.819041 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:27.819022    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-b8fcf97b4-27xgf"
Sep 09 12:13:30.483358 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.483316    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79d6dfdbb4-b8dqn" podUID="8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" containerName="console" containerID="cri-o://f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c" gracePeriod=15
Sep 09 12:13:30.722772 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.722747    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:13:30.854085 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.854054    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-serving-cert\") pod \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") "
Sep 09 12:13:30.854256 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.854120    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2q57\" (UniqueName: \"kubernetes.io/projected/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-kube-api-access-m2q57\") pod \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") "
Sep 09 12:13:30.854256 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.854136    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-oauth-serving-cert\") pod \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") "
Sep 09 12:13:30.854256 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.854160    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-config\") pod \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") "
Sep 09 12:13:30.854256 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.854179    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-oauth-config\") pod \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") "
Sep 09 12:13:30.854256 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.854199    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-service-ca\") pod \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\" (UID: \"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8\") "
Sep 09 12:13:30.854607 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.854578    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-config" (OuterVolumeSpecName: "console-config") pod "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" (UID: "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:13:30.854726 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.854600    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" (UID: "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:13:30.854726 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.854621    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-service-ca" (OuterVolumeSpecName: "service-ca") pod "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" (UID: "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:13:30.856402 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.856376    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" (UID: "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:13:30.856538 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.856427    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" (UID: "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:13:30.856538 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.856515    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-kube-api-access-m2q57" (OuterVolumeSpecName: "kube-api-access-m2q57") pod "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" (UID: "8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8"). InnerVolumeSpecName "kube-api-access-m2q57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:13:30.954976 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.954946    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2q57\" (UniqueName: \"kubernetes.io/projected/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-kube-api-access-m2q57\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:30.954976 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.954971    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-oauth-serving-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:30.954976 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.954982    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-config\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:30.955172 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.954992    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-oauth-config\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:30.955172 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.955001    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:30.955172 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:30.955008    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8-console-serving-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:13:31.544817 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:31.544786    2585 generic.go:356] "Generic (PLEG): container finished" podID="8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" containerID="f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c" exitCode=0
Sep 09 12:13:31.545279 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:31.544834    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d6dfdbb4-b8dqn"
Sep 09 12:13:31.545279 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:31.544872    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d6dfdbb4-b8dqn" event={"ID":"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8","Type":"ContainerDied","Data":"f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c"}
Sep 09 12:13:31.545279 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:31.544930    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d6dfdbb4-b8dqn" event={"ID":"8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8","Type":"ContainerDied","Data":"0a4554bb9a8c2e7b5648f421be9847e5542f01e3271622f67a4318d8db62f4ea"}
Sep 09 12:13:31.545279 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:31.544961    2585 scope.go:117] "RemoveContainer" containerID="f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c"
Sep 09 12:13:31.553673 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:31.553659    2585 scope.go:117] "RemoveContainer" containerID="f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c"
Sep 09 12:13:31.553933 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:13:31.553913    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c\": container with ID starting with f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c not found: ID does not exist" containerID="f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c"
Sep 09 12:13:31.553993 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:31.553943    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c"} err="failed to get container status \"f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c\": rpc error: code = NotFound desc = could not find container \"f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c\": container with ID starting with f4880ee705077c3b10489b55a73133da62595c025fd63a69a8df2789dcb4ff1c not found: ID does not exist"
Sep 09 12:13:31.566907 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:31.566881    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79d6dfdbb4-b8dqn"]
Sep 09 12:13:31.571243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:31.571224    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79d6dfdbb4-b8dqn"]
Sep 09 12:13:31.898914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:31.898839    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" path="/var/lib/kubelet/pods/8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8/volumes"
Sep 09 12:13:49.714922 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:49.714889    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:13:49.733961 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:49.733933    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:13:50.638328 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:13:50.638304    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:03.487364 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.487328    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:14:03.487909 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.487610    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="alertmanager" containerID="cri-o://565fea6c2d81b474081a06541c7057fe64b4bd676867214b343923bea70a02fb" gracePeriod=120
Sep 09 12:14:03.487909 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.487629    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="prom-label-proxy" containerID="cri-o://227d809c7223ae93765f8e3e91746b27641203cd9663780eb90ff2c2e3e0e981" gracePeriod=120
Sep 09 12:14:03.487909 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.487670    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy" containerID="cri-o://47c13d3cfcc7406004634dd3bb4e00db543d0255bf7daf04c86288174187c4ce" gracePeriod=120
Sep 09 12:14:03.487909 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.487678    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy-web" containerID="cri-o://3ec1670c6c45f163bab8a1f78df687d5448fc7e7bce39f680d2524c45b9e7347" gracePeriod=120
Sep 09 12:14:03.487909 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.487719    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="config-reloader" containerID="cri-o://d926857dc2005b460ca5eb1f46b487f57ab91d8c4a1542d5fe20b7356d29d75a" gracePeriod=120
Sep 09 12:14:03.487909 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.487734    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy-metric" containerID="cri-o://bc60f8eba60422f13b5c433e2968783db44276c6a9bf674d18709585f30a33ed" gracePeriod=120
Sep 09 12:14:03.676610 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.676576    2585 generic.go:356] "Generic (PLEG): container finished" podID="63b16436-49e3-451f-8559-51f86adf2fb9" containerID="227d809c7223ae93765f8e3e91746b27641203cd9663780eb90ff2c2e3e0e981" exitCode=0
Sep 09 12:14:03.676610 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.676603    2585 generic.go:356] "Generic (PLEG): container finished" podID="63b16436-49e3-451f-8559-51f86adf2fb9" containerID="bc60f8eba60422f13b5c433e2968783db44276c6a9bf674d18709585f30a33ed" exitCode=0
Sep 09 12:14:03.676811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.676617    2585 generic.go:356] "Generic (PLEG): container finished" podID="63b16436-49e3-451f-8559-51f86adf2fb9" containerID="47c13d3cfcc7406004634dd3bb4e00db543d0255bf7daf04c86288174187c4ce" exitCode=0
Sep 09 12:14:03.676811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.676625    2585 generic.go:356] "Generic (PLEG): container finished" podID="63b16436-49e3-451f-8559-51f86adf2fb9" containerID="d926857dc2005b460ca5eb1f46b487f57ab91d8c4a1542d5fe20b7356d29d75a" exitCode=0
Sep 09 12:14:03.676811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.676633    2585 generic.go:356] "Generic (PLEG): container finished" podID="63b16436-49e3-451f-8559-51f86adf2fb9" containerID="565fea6c2d81b474081a06541c7057fe64b4bd676867214b343923bea70a02fb" exitCode=0
Sep 09 12:14:03.676811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.676643    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerDied","Data":"227d809c7223ae93765f8e3e91746b27641203cd9663780eb90ff2c2e3e0e981"}
Sep 09 12:14:03.676811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.676747    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerDied","Data":"bc60f8eba60422f13b5c433e2968783db44276c6a9bf674d18709585f30a33ed"}
Sep 09 12:14:03.676811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.676774    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerDied","Data":"47c13d3cfcc7406004634dd3bb4e00db543d0255bf7daf04c86288174187c4ce"}
Sep 09 12:14:03.676811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.676800    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerDied","Data":"d926857dc2005b460ca5eb1f46b487f57ab91d8c4a1542d5fe20b7356d29d75a"}
Sep 09 12:14:03.677012 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:03.676823    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerDied","Data":"565fea6c2d81b474081a06541c7057fe64b4bd676867214b343923bea70a02fb"}
Sep 09 12:14:04.684216 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.684187    2585 generic.go:356] "Generic (PLEG): container finished" podID="63b16436-49e3-451f-8559-51f86adf2fb9" containerID="3ec1670c6c45f163bab8a1f78df687d5448fc7e7bce39f680d2524c45b9e7347" exitCode=0
Sep 09 12:14:04.684601 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.684241    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerDied","Data":"3ec1670c6c45f163bab8a1f78df687d5448fc7e7bce39f680d2524c45b9e7347"}
Sep 09 12:14:04.739041 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.739012    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:04.821779 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.821678    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-main-db\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.821779 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.821760    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-cluster-tls-config\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822006 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.821791    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-trusted-ca-bundle\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822006 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.821824    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-web\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822006 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.821856    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-config-volume\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822006 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.821981    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-metrics-client-ca\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822208 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822059    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-web-config\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822208 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822119    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs96m\" (UniqueName: \"kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-kube-api-access-zs96m\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822208 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822162    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-tls-assets\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822208 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822196    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822221    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-main-tls\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822273    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-config-out\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822276    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:14:04.822406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822307    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"63b16436-49e3-451f-8559-51f86adf2fb9\" (UID: \"63b16436-49e3-451f-8559-51f86adf2fb9\") "
Sep 09 12:14:04.822406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822353    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:14:04.822650 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822628    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:14:04.822796 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822773    2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-main-db\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.822863 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822802    2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.822863 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.822817    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63b16436-49e3-451f-8559-51f86adf2fb9-metrics-client-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.825205 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.825178    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:04.825326 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.825271    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:04.825956 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.825931    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-config-volume" (OuterVolumeSpecName: "config-volume") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:04.827196 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.827077    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:04.827196 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.827164    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:04.827373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.827348    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-kube-api-access-zs96m" (OuterVolumeSpecName: "kube-api-access-zs96m") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "kube-api-access-zs96m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:14:04.827437 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.827416    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:14:04.827659 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.827633    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-config-out" (OuterVolumeSpecName: "config-out") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:14:04.830591 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.830411    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:04.836814 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.836793    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-web-config" (OuterVolumeSpecName: "web-config") pod "63b16436-49e3-451f-8559-51f86adf2fb9" (UID: "63b16436-49e3-451f-8559-51f86adf2fb9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:04.923830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.923786    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs96m\" (UniqueName: \"kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-kube-api-access-zs96m\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.923830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.923826    2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/63b16436-49e3-451f-8559-51f86adf2fb9-tls-assets\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.923997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.923844    2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.923997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.923860    2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-main-tls\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.923997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.923873    2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/63b16436-49e3-451f-8559-51f86adf2fb9-config-out\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.923997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.923888    2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.923997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.923901    2585 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-cluster-tls-config\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.923997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.923914    2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.923997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.923927    2585 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-config-volume\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:04.923997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:04.923940    2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/63b16436-49e3-451f-8559-51f86adf2fb9-web-config\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:05.691497 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.691464    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63b16436-49e3-451f-8559-51f86adf2fb9","Type":"ContainerDied","Data":"298fd9a3a9719a43ca407bd91687adc4aa5bba9dfe4282fbf18a239bb0a1c10c"}
Sep 09 12:14:05.691967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.691538    2585 scope.go:117] "RemoveContainer" containerID="227d809c7223ae93765f8e3e91746b27641203cd9663780eb90ff2c2e3e0e981"
Sep 09 12:14:05.691967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.691545    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.699222 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.699196    2585 scope.go:117] "RemoveContainer" containerID="bc60f8eba60422f13b5c433e2968783db44276c6a9bf674d18709585f30a33ed"
Sep 09 12:14:05.706463 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.706444    2585 scope.go:117] "RemoveContainer" containerID="47c13d3cfcc7406004634dd3bb4e00db543d0255bf7daf04c86288174187c4ce"
Sep 09 12:14:05.713786 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.713765    2585 scope.go:117] "RemoveContainer" containerID="3ec1670c6c45f163bab8a1f78df687d5448fc7e7bce39f680d2524c45b9e7347"
Sep 09 12:14:05.718420 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.718395    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:14:05.722837 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.722818    2585 scope.go:117] "RemoveContainer" containerID="d926857dc2005b460ca5eb1f46b487f57ab91d8c4a1542d5fe20b7356d29d75a"
Sep 09 12:14:05.723245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.723223    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:14:05.730298 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.730279    2585 scope.go:117] "RemoveContainer" containerID="565fea6c2d81b474081a06541c7057fe64b4bd676867214b343923bea70a02fb"
Sep 09 12:14:05.737448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.737423    2585 scope.go:117] "RemoveContainer" containerID="f947608ef629ca3e38ab5539c88e284ade9b12fff38d8c4a6bd4306809faf1b3"
Sep 09 12:14:05.750979 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.750953    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:14:05.751231 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751220    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="config-reloader"
Sep 09 12:14:05.751271 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751233    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="config-reloader"
Sep 09 12:14:05.751271 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751243    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy-web"
Sep 09 12:14:05.751271 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751265    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy-web"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751273    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="alertmanager"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751279    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="alertmanager"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751284    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751289    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751299    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" containerName="console"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751304    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" containerName="console"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751312    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="init-config-reloader"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751318    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="init-config-reloader"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751338    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy-metric"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751343    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy-metric"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751350    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6955fe65-c931-4735-8315-03f05ae37092" containerName="registry"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751355    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6955fe65-c931-4735-8315-03f05ae37092" containerName="registry"
Sep 09 12:14:05.751360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751364    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="prom-label-proxy"
Sep 09 12:14:05.751728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751369    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="prom-label-proxy"
Sep 09 12:14:05.751728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751416    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f743664-ab1a-49f3-ba0d-d2d3d76ba3b8" containerName="console"
Sep 09 12:14:05.751728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751422    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy"
Sep 09 12:14:05.751728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751428    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="prom-label-proxy"
Sep 09 12:14:05.751728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751434    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="alertmanager"
Sep 09 12:14:05.751728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751440    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="config-reloader"
Sep 09 12:14:05.751728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751447    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy-metric"
Sep 09 12:14:05.751728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751453    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6955fe65-c931-4735-8315-03f05ae37092" containerName="registry"
Sep 09 12:14:05.751728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.751457    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" containerName="kube-rbac-proxy-web"
Sep 09 12:14:05.756185 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.756167    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.758712 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.758676    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 09 12:14:05.758807 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.758676    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 09 12:14:05.758807 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.758723    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 09 12:14:05.758975 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.758962    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ncnd6\""
Sep 09 12:14:05.759019 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.758991    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 09 12:14:05.759122 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.759109    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 09 12:14:05.759182 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.759151    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 09 12:14:05.759182 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.759172    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 09 12:14:05.759290 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.759174    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 09 12:14:05.765006 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.764991    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 09 12:14:05.768918 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.768899    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:14:05.830277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830183    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g97vf\" (UniqueName: \"kubernetes.io/projected/870b927b-7eea-4d47-acc6-aef50a4d7d99-kube-api-access-g97vf\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830227    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-config-volume\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830257    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830521 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830326    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/870b927b-7eea-4d47-acc6-aef50a4d7d99-config-out\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830521 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830365    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830521 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830385    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/870b927b-7eea-4d47-acc6-aef50a4d7d99-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830521 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830405    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/870b927b-7eea-4d47-acc6-aef50a4d7d99-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830521 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830429    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830521 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830453    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/870b927b-7eea-4d47-acc6-aef50a4d7d99-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830521 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830473    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830777 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830558    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830777 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830588    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/870b927b-7eea-4d47-acc6-aef50a4d7d99-tls-assets\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.830777 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.830608    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-web-config\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.902479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.902451    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63b16436-49e3-451f-8559-51f86adf2fb9" path="/var/lib/kubelet/pods/63b16436-49e3-451f-8559-51f86adf2fb9/volumes"
Sep 09 12:14:05.931578 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931550    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931587    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/870b927b-7eea-4d47-acc6-aef50a4d7d99-tls-assets\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931606    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-web-config\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931640    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g97vf\" (UniqueName: \"kubernetes.io/projected/870b927b-7eea-4d47-acc6-aef50a4d7d99-kube-api-access-g97vf\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931671    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-config-volume\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931719    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931750    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/870b927b-7eea-4d47-acc6-aef50a4d7d99-config-out\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931781    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931805    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/870b927b-7eea-4d47-acc6-aef50a4d7d99-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931828    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/870b927b-7eea-4d47-acc6-aef50a4d7d99-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931864    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931908    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/870b927b-7eea-4d47-acc6-aef50a4d7d99-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.931941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.931939    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.933078 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.933051    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/870b927b-7eea-4d47-acc6-aef50a4d7d99-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.933554 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.933533    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/870b927b-7eea-4d47-acc6-aef50a4d7d99-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.935260 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.934997    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/870b927b-7eea-4d47-acc6-aef50a4d7d99-tls-assets\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.935260 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.935117    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.935260 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.935211    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-web-config\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.935260 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.935211    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.935542 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.935269    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.935542 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.935435    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-config-volume\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.935693 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.935673    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/870b927b-7eea-4d47-acc6-aef50a4d7d99-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.935976 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.935955    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/870b927b-7eea-4d47-acc6-aef50a4d7d99-config-out\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.936095 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.936073    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.937228 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.937211    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/870b927b-7eea-4d47-acc6-aef50a4d7d99-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:05.945796 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:05.945772    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g97vf\" (UniqueName: \"kubernetes.io/projected/870b927b-7eea-4d47-acc6-aef50a4d7d99-kube-api-access-g97vf\") pod \"alertmanager-main-0\" (UID: \"870b927b-7eea-4d47-acc6-aef50a4d7d99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:06.065758 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:06.065723    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 12:14:06.351662 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:06.351639    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 12:14:06.353063 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:14:06.353043    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod870b927b_7eea_4d47_acc6_aef50a4d7d99.slice/crio-9a5a224b80893285fd21bcd38120ea12a7e4e85fdc7c4bb97e80528ceccb242b WatchSource:0}: Error finding container 9a5a224b80893285fd21bcd38120ea12a7e4e85fdc7c4bb97e80528ceccb242b: Status 404 returned error can't find the container with id 9a5a224b80893285fd21bcd38120ea12a7e4e85fdc7c4bb97e80528ceccb242b
Sep 09 12:14:06.698915 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:06.698823    2585 generic.go:356] "Generic (PLEG): container finished" podID="870b927b-7eea-4d47-acc6-aef50a4d7d99" containerID="50c4417a255a8f4dfb00a95d2495c94bc91c0c41616df39d68ab067ea2eb6227" exitCode=0
Sep 09 12:14:06.698915 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:06.698859    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"870b927b-7eea-4d47-acc6-aef50a4d7d99","Type":"ContainerDied","Data":"50c4417a255a8f4dfb00a95d2495c94bc91c0c41616df39d68ab067ea2eb6227"}
Sep 09 12:14:06.698915 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:06.698913    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"870b927b-7eea-4d47-acc6-aef50a4d7d99","Type":"ContainerStarted","Data":"9a5a224b80893285fd21bcd38120ea12a7e4e85fdc7c4bb97e80528ceccb242b"}
Sep 09 12:14:07.540162 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.540119    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-dfc4d978c-58tq2"]
Sep 09 12:14:07.543542 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.543519    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.545972 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.545945    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 09 12:14:07.546068 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.545958    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 09 12:14:07.546150 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.546132    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lmw78\""
Sep 09 12:14:07.546352 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.546333    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 09 12:14:07.546409 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.546333    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 09 12:14:07.547184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.547169    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 09 12:14:07.550863 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.550842    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 09 12:14:07.558387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.558365    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-dfc4d978c-58tq2"]
Sep 09 12:14:07.645627 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.645588    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f45db69e-4fd1-478e-9243-aebfc9a51c50-metrics-client-ca\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.645627 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.645633    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxgpl\" (UniqueName: \"kubernetes.io/projected/f45db69e-4fd1-478e-9243-aebfc9a51c50-kube-api-access-lxgpl\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.645910 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.645755    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45db69e-4fd1-478e-9243-aebfc9a51c50-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.645910 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.645793    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45db69e-4fd1-478e-9243-aebfc9a51c50-serving-certs-ca-bundle\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.645910 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.645817    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-telemeter-client-tls\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.645910 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.645874    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-secret-telemeter-client\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.646080 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.645919    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.646080 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.645948    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-federate-client-tls\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.706840 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.706804    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"870b927b-7eea-4d47-acc6-aef50a4d7d99","Type":"ContainerStarted","Data":"3b594a1b236db8348151a0356ad64eb41cf69a778082509e0274082ba016a340"}
Sep 09 12:14:07.707291 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.706879    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"870b927b-7eea-4d47-acc6-aef50a4d7d99","Type":"ContainerStarted","Data":"3f902b1d768ecf6f0dd9dcca9108ec85e9ec5816388b9cb17faa9a995e68de39"}
Sep 09 12:14:07.707291 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.706920    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"870b927b-7eea-4d47-acc6-aef50a4d7d99","Type":"ContainerStarted","Data":"1ee6505a1f1a8295203589527792cb3e1f6c6cddd471054108ae6c89245bbb72"}
Sep 09 12:14:07.707291 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.706961    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"870b927b-7eea-4d47-acc6-aef50a4d7d99","Type":"ContainerStarted","Data":"836753e2e5a6be97a9c91bef761119a60fba68a217246e185b9579ead9fbbbb2"}
Sep 09 12:14:07.707291 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.707007    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"870b927b-7eea-4d47-acc6-aef50a4d7d99","Type":"ContainerStarted","Data":"d00596ed87e2204c0807d139016a20005b41cba62004a8246e7adfadd5524507"}
Sep 09 12:14:07.707291 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.707045    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"870b927b-7eea-4d47-acc6-aef50a4d7d99","Type":"ContainerStarted","Data":"1dff7df00fe34d7bf8f8419175dd7d2830bf835371479e2f8b110282099f9c56"}
Sep 09 12:14:07.739550 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.739464    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.7239377 podStartE2EDuration="2.739445508s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.507808ms" isStatefulPod=false podCreationTimestamp="2026-09-09 12:14:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:14:07.737454084 +0000 UTC m=+192.379073322" watchObservedRunningTime="2026-09-09 12:14:07.739445508 +0000 UTC m=+192.381064713"
Sep 09 12:14:07.746629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.746605    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45db69e-4fd1-478e-9243-aebfc9a51c50-serving-certs-ca-bundle\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.746778 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.746633    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-telemeter-client-tls\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.746778 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.746751    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-secret-telemeter-client\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.746897 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.746783    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.746897 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.746809    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-federate-client-tls\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.746897 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.746865    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f45db69e-4fd1-478e-9243-aebfc9a51c50-metrics-client-ca\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.747108 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.746900    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxgpl\" (UniqueName: \"kubernetes.io/projected/f45db69e-4fd1-478e-9243-aebfc9a51c50-kube-api-access-lxgpl\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.747108 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.747064    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45db69e-4fd1-478e-9243-aebfc9a51c50-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.748387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.747965    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45db69e-4fd1-478e-9243-aebfc9a51c50-serving-certs-ca-bundle\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.748387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.748092    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f45db69e-4fd1-478e-9243-aebfc9a51c50-metrics-client-ca\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.748387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.748339    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f45db69e-4fd1-478e-9243-aebfc9a51c50-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.750401 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.750071    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.750401 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.750241    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-secret-telemeter-client\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.750401 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.750340    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-telemeter-client-tls\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.750401 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.750349    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f45db69e-4fd1-478e-9243-aebfc9a51c50-federate-client-tls\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.759335 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.759311    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxgpl\" (UniqueName: \"kubernetes.io/projected/f45db69e-4fd1-478e-9243-aebfc9a51c50-kube-api-access-lxgpl\") pod \"telemeter-client-dfc4d978c-58tq2\" (UID: \"f45db69e-4fd1-478e-9243-aebfc9a51c50\") " pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:07.828282 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.828207    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 12:14:07.828544 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.828495    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="prometheus" containerID="cri-o://2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5" gracePeriod=600
Sep 09 12:14:07.828544 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.828530    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c" gracePeriod=600
Sep 09 12:14:07.828758 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.828543    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy-web" containerID="cri-o://31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a" gracePeriod=600
Sep 09 12:14:07.828758 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.828546    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="thanos-sidecar" containerID="cri-o://67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a" gracePeriod=600
Sep 09 12:14:07.828758 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.828516    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy" containerID="cri-o://9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b" gracePeriod=600
Sep 09 12:14:07.828758 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.828548    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="config-reloader" containerID="cri-o://73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a" gracePeriod=600
Sep 09 12:14:07.853218 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:07.853197    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2"
Sep 09 12:14:08.070539 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.070509    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.141638 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.141564    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-dfc4d978c-58tq2"]
Sep 09 12:14:08.145077 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:14:08.145051    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45db69e_4fd1_478e_9243_aebfc9a51c50.slice/crio-602f50c57ba0b382a63278cb40424f4e1062d5419924b1b5c5161ef35ac95c16 WatchSource:0}: Error finding container 602f50c57ba0b382a63278cb40424f4e1062d5419924b1b5c5161ef35ac95c16: Status 404 returned error can't find the container with id 602f50c57ba0b382a63278cb40424f4e1062d5419924b1b5c5161ef35ac95c16
Sep 09 12:14:08.150796 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.150777    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-tls\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.150877 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.150816    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-grpc-tls\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.150877 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.150840    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-kubelet-serving-ca-bundle\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.150877 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.150864    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-serving-certs-ca-bundle\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151044 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151025    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-db\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151100 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151078    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-tls-assets\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151153 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151125    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-1\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151212 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151159    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-kube-rbac-proxy\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151212 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151182    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:14:08.151212 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151190    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-thanos-prometheus-http-client-file\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151219    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:14:08.151347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151225    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbn7w\" (UniqueName: \"kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-kube-api-access-dbn7w\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151352    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-0\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151386    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-metrics-client-ca\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151410    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:14:08.151448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151423    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config-out\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151448    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-metrics-client-certs\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151479    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151508    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-2\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151531    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-web-config\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151581    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151599    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.151636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151613    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-trusted-ca-bundle\") pod \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\" (UID: \"85cb5b63-1148-422f-b4f2-b0b7c0664e1d\") "
Sep 09 12:14:08.152008 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.151875    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.152633 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.152605    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.152737 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.152638    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.153097 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.153072    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:14:08.153454 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.153423    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:14:08.153586 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.153560    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:14:08.153742 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.153684    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:14:08.154346 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.154323    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:14:08.154431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.154324    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:08.154431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.154369    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:08.154973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.154948    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:14:08.155452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.155424    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:08.156005 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.155969    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-kube-api-access-dbn7w" (OuterVolumeSpecName: "kube-api-access-dbn7w") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "kube-api-access-dbn7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:14:08.156581 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.156550    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config" (OuterVolumeSpecName: "config") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:08.156581 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.156576    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:08.156767 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.156598    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:08.156767 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.156614    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:08.157315 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.157296    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config-out" (OuterVolumeSpecName: "config-out") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:14:08.157584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.157564    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:08.168555 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.168534    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-web-config" (OuterVolumeSpecName: "web-config") pod "85cb5b63-1148-422f-b4f2-b0b7c0664e1d" (UID: "85cb5b63-1148-422f-b4f2-b0b7c0664e1d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:14:08.253758 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253720    2585 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-kube-rbac-proxy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253758 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253752    2585 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253758 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253762    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbn7w\" (UniqueName: \"kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-kube-api-access-dbn7w\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253773    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253782    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-configmap-metrics-client-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253791    2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config-out\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253799    2585 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-metrics-client-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253809    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253817    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253825    2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-web-config\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253833    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253841    2585 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-config\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253849    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253857    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253864    2585 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-secret-grpc-tls\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253873    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-prometheus-k8s-db\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.253973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.253880    2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85cb5b63-1148-422f-b4f2-b0b7c0664e1d-tls-assets\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:14:08.711946 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.711911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2" event={"ID":"f45db69e-4fd1-478e-9243-aebfc9a51c50","Type":"ContainerStarted","Data":"602f50c57ba0b382a63278cb40424f4e1062d5419924b1b5c5161ef35ac95c16"}
Sep 09 12:14:08.714943 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.714916    2585 generic.go:356] "Generic (PLEG): container finished" podID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerID="c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c" exitCode=0
Sep 09 12:14:08.714943 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.714942    2585 generic.go:356] "Generic (PLEG): container finished" podID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerID="9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b" exitCode=0
Sep 09 12:14:08.715112 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.714953    2585 generic.go:356] "Generic (PLEG): container finished" podID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerID="31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a" exitCode=0
Sep 09 12:14:08.715112 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.714961    2585 generic.go:356] "Generic (PLEG): container finished" podID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerID="67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a" exitCode=0
Sep 09 12:14:08.715112 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.714967    2585 generic.go:356] "Generic (PLEG): container finished" podID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerID="73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a" exitCode=0
Sep 09 12:14:08.715112 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.714974    2585 generic.go:356] "Generic (PLEG): container finished" podID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerID="2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5" exitCode=0
Sep 09 12:14:08.715112 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.714991    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.715112 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.715007    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerDied","Data":"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"}
Sep 09 12:14:08.715112 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.715069    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerDied","Data":"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"}
Sep 09 12:14:08.715112 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.715096    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerDied","Data":"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"}
Sep 09 12:14:08.715436 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.715123    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerDied","Data":"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"}
Sep 09 12:14:08.715436 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.715138    2585 scope.go:117] "RemoveContainer" containerID="c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"
Sep 09 12:14:08.715436 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.715148    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerDied","Data":"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"}
Sep 09 12:14:08.715436 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.715171    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerDied","Data":"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"}
Sep 09 12:14:08.715436 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.715193    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"85cb5b63-1148-422f-b4f2-b0b7c0664e1d","Type":"ContainerDied","Data":"3207cfd956c418846c4cdc10964388eac195c19ca7e178f3f7f992b45368c0df"}
Sep 09 12:14:08.723596 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.723573    2585 scope.go:117] "RemoveContainer" containerID="9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"
Sep 09 12:14:08.730332 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.730313    2585 scope.go:117] "RemoveContainer" containerID="31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"
Sep 09 12:14:08.737062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.737048    2585 scope.go:117] "RemoveContainer" containerID="67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"
Sep 09 12:14:08.742312 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.742294    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 12:14:08.744625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.744608    2585 scope.go:117] "RemoveContainer" containerID="73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"
Sep 09 12:14:08.747891 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.747869    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 12:14:08.751432 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.751413    2585 scope.go:117] "RemoveContainer" containerID="2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"
Sep 09 12:14:08.758736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.758717    2585 scope.go:117] "RemoveContainer" containerID="e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"
Sep 09 12:14:08.765116 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.765101    2585 scope.go:117] "RemoveContainer" containerID="c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"
Sep 09 12:14:08.765396 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:14:08.765378    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": container with ID starting with c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c not found: ID does not exist" containerID="c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"
Sep 09 12:14:08.765471 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.765404    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"} err="failed to get container status \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": rpc error: code = NotFound desc = could not find container \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": container with ID starting with c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c not found: ID does not exist"
Sep 09 12:14:08.765471 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.765422    2585 scope.go:117] "RemoveContainer" containerID="9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"
Sep 09 12:14:08.765658 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:14:08.765642    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": container with ID starting with 9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b not found: ID does not exist" containerID="9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"
Sep 09 12:14:08.765693 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.765673    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"} err="failed to get container status \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": rpc error: code = NotFound desc = could not find container \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": container with ID starting with 9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b not found: ID does not exist"
Sep 09 12:14:08.765693 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.765686    2585 scope.go:117] "RemoveContainer" containerID="31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"
Sep 09 12:14:08.765959 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:14:08.765943    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": container with ID starting with 31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a not found: ID does not exist" containerID="31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"
Sep 09 12:14:08.766008 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.765963    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"} err="failed to get container status \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": rpc error: code = NotFound desc = could not find container \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": container with ID starting with 31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a not found: ID does not exist"
Sep 09 12:14:08.766008 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.765976    2585 scope.go:117] "RemoveContainer" containerID="67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"
Sep 09 12:14:08.766222 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:14:08.766206    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": container with ID starting with 67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a not found: ID does not exist" containerID="67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"
Sep 09 12:14:08.766262 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.766226    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"} err="failed to get container status \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": rpc error: code = NotFound desc = could not find container \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": container with ID starting with 67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a not found: ID does not exist"
Sep 09 12:14:08.766262 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.766238    2585 scope.go:117] "RemoveContainer" containerID="73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"
Sep 09 12:14:08.766471 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:14:08.766457    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": container with ID starting with 73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a not found: ID does not exist" containerID="73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"
Sep 09 12:14:08.766509 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.766474    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"} err="failed to get container status \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": rpc error: code = NotFound desc = could not find container \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": container with ID starting with 73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a not found: ID does not exist"
Sep 09 12:14:08.766509 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.766484    2585 scope.go:117] "RemoveContainer" containerID="2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"
Sep 09 12:14:08.766680 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:14:08.766663    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": container with ID starting with 2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5 not found: ID does not exist" containerID="2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"
Sep 09 12:14:08.766741 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.766686    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"} err="failed to get container status \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": rpc error: code = NotFound desc = could not find container \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": container with ID starting with 2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5 not found: ID does not exist"
Sep 09 12:14:08.766741 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.766717    2585 scope.go:117] "RemoveContainer" containerID="e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"
Sep 09 12:14:08.766959 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:14:08.766944    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": container with ID starting with e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07 not found: ID does not exist" containerID="e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"
Sep 09 12:14:08.766998 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.766962    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"} err="failed to get container status \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": rpc error: code = NotFound desc = could not find container \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": container with ID starting with e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07 not found: ID does not exist"
Sep 09 12:14:08.766998 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.766973    2585 scope.go:117] "RemoveContainer" containerID="c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"
Sep 09 12:14:08.767164 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.767149    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"} err="failed to get container status \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": rpc error: code = NotFound desc = could not find container \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": container with ID starting with c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c not found: ID does not exist"
Sep 09 12:14:08.767205 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.767164    2585 scope.go:117] "RemoveContainer" containerID="9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"
Sep 09 12:14:08.767385 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.767368    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"} err="failed to get container status \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": rpc error: code = NotFound desc = could not find container \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": container with ID starting with 9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b not found: ID does not exist"
Sep 09 12:14:08.767432 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.767385    2585 scope.go:117] "RemoveContainer" containerID="31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"
Sep 09 12:14:08.767601 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.767585    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"} err="failed to get container status \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": rpc error: code = NotFound desc = could not find container \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": container with ID starting with 31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a not found: ID does not exist"
Sep 09 12:14:08.767639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.767601    2585 scope.go:117] "RemoveContainer" containerID="67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"
Sep 09 12:14:08.767846 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.767829    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"} err="failed to get container status \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": rpc error: code = NotFound desc = could not find container \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": container with ID starting with 67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a not found: ID does not exist"
Sep 09 12:14:08.767891 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.767846    2585 scope.go:117] "RemoveContainer" containerID="73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"
Sep 09 12:14:08.768083 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.768066    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"} err="failed to get container status \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": rpc error: code = NotFound desc = could not find container \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": container with ID starting with 73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a not found: ID does not exist"
Sep 09 12:14:08.768130 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.768084    2585 scope.go:117] "RemoveContainer" containerID="2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"
Sep 09 12:14:08.768306 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.768287    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"} err="failed to get container status \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": rpc error: code = NotFound desc = could not find container \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": container with ID starting with 2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5 not found: ID does not exist"
Sep 09 12:14:08.768360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.768306    2585 scope.go:117] "RemoveContainer" containerID="e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"
Sep 09 12:14:08.768534 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.768517    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"} err="failed to get container status \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": rpc error: code = NotFound desc = could not find container \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": container with ID starting with e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07 not found: ID does not exist"
Sep 09 12:14:08.768575 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.768534    2585 scope.go:117] "RemoveContainer" containerID="c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"
Sep 09 12:14:08.768777 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.768760    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"} err="failed to get container status \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": rpc error: code = NotFound desc = could not find container \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": container with ID starting with c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c not found: ID does not exist"
Sep 09 12:14:08.768826 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.768777    2585 scope.go:117] "RemoveContainer" containerID="9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"
Sep 09 12:14:08.769002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.768986    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"} err="failed to get container status \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": rpc error: code = NotFound desc = could not find container \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": container with ID starting with 9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b not found: ID does not exist"
Sep 09 12:14:08.769059 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.769002    2585 scope.go:117] "RemoveContainer" containerID="31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"
Sep 09 12:14:08.769222 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.769205    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"} err="failed to get container status \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": rpc error: code = NotFound desc = could not find container \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": container with ID starting with 31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a not found: ID does not exist"
Sep 09 12:14:08.769266 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.769222    2585 scope.go:117] "RemoveContainer" containerID="67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"
Sep 09 12:14:08.769447 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.769432    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"} err="failed to get container status \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": rpc error: code = NotFound desc = could not find container \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": container with ID starting with 67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a not found: ID does not exist"
Sep 09 12:14:08.769447 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.769447    2585 scope.go:117] "RemoveContainer" containerID="73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"
Sep 09 12:14:08.769677 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.769661    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"} err="failed to get container status \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": rpc error: code = NotFound desc = could not find container \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": container with ID starting with 73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a not found: ID does not exist"
Sep 09 12:14:08.769734 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.769678    2585 scope.go:117] "RemoveContainer" containerID="2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"
Sep 09 12:14:08.769914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.769893    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"} err="failed to get container status \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": rpc error: code = NotFound desc = could not find container \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": container with ID starting with 2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5 not found: ID does not exist"
Sep 09 12:14:08.769964 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.769916    2585 scope.go:117] "RemoveContainer" containerID="e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"
Sep 09 12:14:08.770141 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.770127    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"} err="failed to get container status \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": rpc error: code = NotFound desc = could not find container \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": container with ID starting with e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07 not found: ID does not exist"
Sep 09 12:14:08.770141 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.770140    2585 scope.go:117] "RemoveContainer" containerID="c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"
Sep 09 12:14:08.770347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.770331    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"} err="failed to get container status \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": rpc error: code = NotFound desc = could not find container \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": container with ID starting with c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c not found: ID does not exist"
Sep 09 12:14:08.770347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.770347    2585 scope.go:117] "RemoveContainer" containerID="9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"
Sep 09 12:14:08.770564 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.770550    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"} err="failed to get container status \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": rpc error: code = NotFound desc = could not find container \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": container with ID starting with 9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b not found: ID does not exist"
Sep 09 12:14:08.770606 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.770563    2585 scope.go:117] "RemoveContainer" containerID="31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"
Sep 09 12:14:08.770806 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.770788    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"} err="failed to get container status \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": rpc error: code = NotFound desc = could not find container \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": container with ID starting with 31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a not found: ID does not exist"
Sep 09 12:14:08.770855 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.770807    2585 scope.go:117] "RemoveContainer" containerID="67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"
Sep 09 12:14:08.771018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.770996    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"} err="failed to get container status \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": rpc error: code = NotFound desc = could not find container \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": container with ID starting with 67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a not found: ID does not exist"
Sep 09 12:14:08.771018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.771018    2585 scope.go:117] "RemoveContainer" containerID="73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"
Sep 09 12:14:08.771276 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.771257    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"} err="failed to get container status \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": rpc error: code = NotFound desc = could not find container \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": container with ID starting with 73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a not found: ID does not exist"
Sep 09 12:14:08.771343 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.771276    2585 scope.go:117] "RemoveContainer" containerID="2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"
Sep 09 12:14:08.771493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.771474    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"} err="failed to get container status \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": rpc error: code = NotFound desc = could not find container \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": container with ID starting with 2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5 not found: ID does not exist"
Sep 09 12:14:08.771493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.771490    2585 scope.go:117] "RemoveContainer" containerID="e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"
Sep 09 12:14:08.771791 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.771768    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"} err="failed to get container status \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": rpc error: code = NotFound desc = could not find container \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": container with ID starting with e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07 not found: ID does not exist"
Sep 09 12:14:08.771875 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.771798    2585 scope.go:117] "RemoveContainer" containerID="c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"
Sep 09 12:14:08.772164 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.772132    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"} err="failed to get container status \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": rpc error: code = NotFound desc = could not find container \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": container with ID starting with c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c not found: ID does not exist"
Sep 09 12:14:08.772245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.772165    2585 scope.go:117] "RemoveContainer" containerID="9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"
Sep 09 12:14:08.772432 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.772415    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"} err="failed to get container status \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": rpc error: code = NotFound desc = could not find container \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": container with ID starting with 9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b not found: ID does not exist"
Sep 09 12:14:08.772484 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.772433    2585 scope.go:117] "RemoveContainer" containerID="31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"
Sep 09 12:14:08.772645 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.772628    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"} err="failed to get container status \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": rpc error: code = NotFound desc = could not find container \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": container with ID starting with 31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a not found: ID does not exist"
Sep 09 12:14:08.772716 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.772645    2585 scope.go:117] "RemoveContainer" containerID="67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"
Sep 09 12:14:08.772903 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.772883    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"} err="failed to get container status \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": rpc error: code = NotFound desc = could not find container \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": container with ID starting with 67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a not found: ID does not exist"
Sep 09 12:14:08.772903 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.772901    2585 scope.go:117] "RemoveContainer" containerID="73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"
Sep 09 12:14:08.773147 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.773130    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"} err="failed to get container status \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": rpc error: code = NotFound desc = could not find container \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": container with ID starting with 73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a not found: ID does not exist"
Sep 09 12:14:08.773218 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.773149    2585 scope.go:117] "RemoveContainer" containerID="2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"
Sep 09 12:14:08.773362 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.773345    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"} err="failed to get container status \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": rpc error: code = NotFound desc = could not find container \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": container with ID starting with 2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5 not found: ID does not exist"
Sep 09 12:14:08.773362 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.773361    2585 scope.go:117] "RemoveContainer" containerID="e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"
Sep 09 12:14:08.773552 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.773538    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"} err="failed to get container status \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": rpc error: code = NotFound desc = could not find container \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": container with ID starting with e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07 not found: ID does not exist"
Sep 09 12:14:08.773590 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.773551    2585 scope.go:117] "RemoveContainer" containerID="c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"
Sep 09 12:14:08.773815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.773797    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c"} err="failed to get container status \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": rpc error: code = NotFound desc = could not find container \"c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c\": container with ID starting with c53dc0ff5ebe0ce67145135fec3ba14505de5156a5693eacb173d655094b893c not found: ID does not exist"
Sep 09 12:14:08.773861 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.773815    2585 scope.go:117] "RemoveContainer" containerID="9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"
Sep 09 12:14:08.774017 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.774001    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b"} err="failed to get container status \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": rpc error: code = NotFound desc = could not find container \"9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b\": container with ID starting with 9d25b6920dbe22060029a2e4e569038e81d44fc6466152e6c98047184382e75b not found: ID does not exist"
Sep 09 12:14:08.774061 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.774017    2585 scope.go:117] "RemoveContainer" containerID="31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"
Sep 09 12:14:08.774211 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.774198    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a"} err="failed to get container status \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": rpc error: code = NotFound desc = could not find container \"31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a\": container with ID starting with 31b4066e2aa21d808b0b140ee44432bb8478e6f8e9c173038bc0b7d365b35c0a not found: ID does not exist"
Sep 09 12:14:08.774250 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.774211    2585 scope.go:117] "RemoveContainer" containerID="67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"
Sep 09 12:14:08.774387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.774372    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a"} err="failed to get container status \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": rpc error: code = NotFound desc = could not find container \"67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a\": container with ID starting with 67542b0ac86e72ce45313e770e2dc273f71108e31cc26fbc4aa24fbc69acbb2a not found: ID does not exist"
Sep 09 12:14:08.774429 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.774388    2585 scope.go:117] "RemoveContainer" containerID="73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"
Sep 09 12:14:08.774632 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.774617    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a"} err="failed to get container status \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": rpc error: code = NotFound desc = could not find container \"73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a\": container with ID starting with 73fe99d3b887a492ca2bd16dd91153eeb4534b5e8cd9ac213b4ea2fcee4cea8a not found: ID does not exist"
Sep 09 12:14:08.774632 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.774630    2585 scope.go:117] "RemoveContainer" containerID="2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"
Sep 09 12:14:08.774870 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.774852    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5"} err="failed to get container status \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": rpc error: code = NotFound desc = could not find container \"2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5\": container with ID starting with 2e1c993e7cc934b6441a7ac8c85f507aae27a66b22c780c7611db028aefabcc5 not found: ID does not exist"
Sep 09 12:14:08.774926 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.774871    2585 scope.go:117] "RemoveContainer" containerID="e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"
Sep 09 12:14:08.775117 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.775099    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07"} err="failed to get container status \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": rpc error: code = NotFound desc = could not find container \"e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07\": container with ID starting with e0fcc1217333454c25ba1b2d006200573f9f6e30083460130d958b16e7223b07 not found: ID does not exist"
Sep 09 12:14:08.782639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.782617    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 12:14:08.782948 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.782934    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="prometheus"
Sep 09 12:14:08.782988 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.782950    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="prometheus"
Sep 09 12:14:08.782988 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.782962    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="thanos-sidecar"
Sep 09 12:14:08.782988 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.782968    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="thanos-sidecar"
Sep 09 12:14:08.782988 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.782975    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="init-config-reloader"
Sep 09 12:14:08.782988 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.782981    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="init-config-reloader"
Sep 09 12:14:08.782988 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.782989    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy-thanos"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.782994    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy-thanos"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783002    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783007    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783019    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy-web"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783024    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy-web"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783030    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="config-reloader"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783035    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="config-reloader"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783079    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="config-reloader"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783085    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783091    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy-thanos"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783098    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="kube-rbac-proxy-web"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783103    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="prometheus"
Sep 09 12:14:08.783154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.783109    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" containerName="thanos-sidecar"
Sep 09 12:14:08.788240 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.788225    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.795652 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.795635    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 09 12:14:08.795764 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.795637    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 09 12:14:08.795764 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.795671    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 09 12:14:08.795866 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.795814    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 09 12:14:08.796263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.796248    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 09 12:14:08.796733 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.796715    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 09 12:14:08.796819 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.796758    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-94fanvck97soo\""
Sep 09 12:14:08.796819 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.796761    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 09 12:14:08.796999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.796985    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 09 12:14:08.797071 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.796990    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 09 12:14:08.797133 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.797094    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 09 12:14:08.797967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.797947    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-22fkk\""
Sep 09 12:14:08.798052 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.797949    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 09 12:14:08.798052 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.798000    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 09 12:14:08.798203 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.798190    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 09 12:14:08.807755 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.807738    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 09 12:14:08.853989 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.853914    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 12:14:08.858770 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.858744    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.858900 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.858779    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.858900 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.858810    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.858900 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.858879    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62b29a26-2b25-4eaa-965e-5f09ed467f11-config-out\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.858923    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62b29a26-2b25-4eaa-965e-5f09ed467f11-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.858943    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.858964    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.858985    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859006    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859039    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-config\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859056    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859098    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-web-config\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859127    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859160    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859198    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859226    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859249    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859286    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzqfv\" (UniqueName: \"kubernetes.io/projected/62b29a26-2b25-4eaa-965e-5f09ed467f11-kube-api-access-kzqfv\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859312    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.859439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.859331    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959778 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959745    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959778 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959779    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959803    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959828    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62b29a26-2b25-4eaa-965e-5f09ed467f11-config-out\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959852    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62b29a26-2b25-4eaa-965e-5f09ed467f11-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959866    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959880    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959898    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959917    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959933    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-config\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959953    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.959994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.959978    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-web-config\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960005    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960038    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960085    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960119    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960147    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960171    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzqfv\" (UniqueName: \"kubernetes.io/projected/62b29a26-2b25-4eaa-965e-5f09ed467f11-kube-api-access-kzqfv\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960197    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960222    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960495    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960534    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960649    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.960886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.960768    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.961167 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.961144    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.961445 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.961411    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.964277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.963677    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.964277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.963802    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.964277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.963847    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62b29a26-2b25-4eaa-965e-5f09ed467f11-config-out\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.964277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.964224    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-config\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.964277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.964229    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.964588 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.964387    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62b29a26-2b25-4eaa-965e-5f09ed467f11-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.965584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.965532    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.965727 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.965620    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.965852 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.965832    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.966205 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.966169    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.966205 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.966186    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/62b29a26-2b25-4eaa-965e-5f09ed467f11-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.966539 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.966520    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-web-config\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.966539 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.966533    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/62b29a26-2b25-4eaa-965e-5f09ed467f11-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:08.978179 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:08.978158    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzqfv\" (UniqueName: \"kubernetes.io/projected/62b29a26-2b25-4eaa-965e-5f09ed467f11-kube-api-access-kzqfv\") pod \"prometheus-k8s-0\" (UID: \"62b29a26-2b25-4eaa-965e-5f09ed467f11\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:09.097249 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:09.097214    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:14:09.389375 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:09.389347    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 12:14:09.391442 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:14:09.391418    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b29a26_2b25_4eaa_965e_5f09ed467f11.slice/crio-bfcd158e3cafd3156810c51a6c047ccc8d179cf185856adebc6d4e74dbc8af60 WatchSource:0}: Error finding container bfcd158e3cafd3156810c51a6c047ccc8d179cf185856adebc6d4e74dbc8af60: Status 404 returned error can't find the container with id bfcd158e3cafd3156810c51a6c047ccc8d179cf185856adebc6d4e74dbc8af60
Sep 09 12:14:09.721942 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:09.721851    2585 generic.go:356] "Generic (PLEG): container finished" podID="62b29a26-2b25-4eaa-965e-5f09ed467f11" containerID="c9873b957c3ba109a00246b6b03b2b3ab70c23d30b5367cf4362a8233d48c714" exitCode=0
Sep 09 12:14:09.722337 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:09.721963    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62b29a26-2b25-4eaa-965e-5f09ed467f11","Type":"ContainerDied","Data":"c9873b957c3ba109a00246b6b03b2b3ab70c23d30b5367cf4362a8233d48c714"}
Sep 09 12:14:09.722337 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:09.722071    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62b29a26-2b25-4eaa-965e-5f09ed467f11","Type":"ContainerStarted","Data":"bfcd158e3cafd3156810c51a6c047ccc8d179cf185856adebc6d4e74dbc8af60"}
Sep 09 12:14:09.899528 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:09.899495    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85cb5b63-1148-422f-b4f2-b0b7c0664e1d" path="/var/lib/kubelet/pods/85cb5b63-1148-422f-b4f2-b0b7c0664e1d/volumes"
Sep 09 12:14:10.731075 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.730986    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62b29a26-2b25-4eaa-965e-5f09ed467f11","Type":"ContainerStarted","Data":"7ad7d4baa62d0cb517ecccdb09b96fc4db839fdb5a4704571841eafc186453ee"}
Sep 09 12:14:10.731075 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.731056    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62b29a26-2b25-4eaa-965e-5f09ed467f11","Type":"ContainerStarted","Data":"668c9e0c1dd73c2fe7afd1a79f75722c32815bd7e55354aa1859e2edac93cb45"}
Sep 09 12:14:10.731517 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.731092    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62b29a26-2b25-4eaa-965e-5f09ed467f11","Type":"ContainerStarted","Data":"15794d7ad245708858085aabb8695a6acc5baa6493c7a3f31b9fb9f4491660f5"}
Sep 09 12:14:10.731517 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.731126    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62b29a26-2b25-4eaa-965e-5f09ed467f11","Type":"ContainerStarted","Data":"a53e85eefce54e6ca3f95fb8eefa559a1824dc7f763ce4d4c85ee3d373e57327"}
Sep 09 12:14:10.731517 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.731160    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62b29a26-2b25-4eaa-965e-5f09ed467f11","Type":"ContainerStarted","Data":"2e44de2ea93f80bfae5fc3ba73caa428cffa796159f5ef1e0746f78f177a271c"}
Sep 09 12:14:10.731517 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.731194    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62b29a26-2b25-4eaa-965e-5f09ed467f11","Type":"ContainerStarted","Data":"46985dab2fca5be32625d2055c5ba94e293084c389a6b3d7bfdb69cb1cfd04aa"}
Sep 09 12:14:10.733136 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.733112    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2" event={"ID":"f45db69e-4fd1-478e-9243-aebfc9a51c50","Type":"ContainerStarted","Data":"98589c3700611d65e0b2111bfcae546e31c45c8679cfecc57ca4ffc8d89e7927"}
Sep 09 12:14:10.733272 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.733155    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2" event={"ID":"f45db69e-4fd1-478e-9243-aebfc9a51c50","Type":"ContainerStarted","Data":"795a9e5c22050c187d8b09269a39d0e8279842877bb7dd4e4733385d9369ec8c"}
Sep 09 12:14:10.733272 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.733175    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2" event={"ID":"f45db69e-4fd1-478e-9243-aebfc9a51c50","Type":"ContainerStarted","Data":"539745f20c5d9f145803fe0b7bdcc430870795e3427922d9eede6c21f5db032c"}
Sep 09 12:14:10.762012 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.761963    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.7378146770000003 podStartE2EDuration="2.761949221s" totalImagesPullingTime="0s" totalInitContainerRuntime="24.134544ms" isStatefulPod=false podCreationTimestamp="2026-09-09 12:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:14:10.761130806 +0000 UTC m=+195.402750004" watchObservedRunningTime="2026-09-09 12:14:10.761949221 +0000 UTC m=+195.403568460"
Sep 09 12:14:10.787943 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:10.787895    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-dfc4d978c-58tq2" podStartSLOduration=1.813109627 podStartE2EDuration="3.787880589s" totalImagesPullingTime="1.974770962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:14:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:14:10.784269707 +0000 UTC m=+195.425888912" watchObservedRunningTime="2026-09-09 12:14:10.787880589 +0000 UTC m=+195.429499793"
Sep 09 12:14:14.097879 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:14:14.097836    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:15:09.097679 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:15:09.097646    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:15:09.113946 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:15:09.113919    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:15:09.999861 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:15:09.999831    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 12:15:55.809763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:15:55.809736    2585 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 12:17:16.508462 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.508427    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"]
Sep 09 12:17:16.511851 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.511835    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"
Sep 09 12:17:16.514195 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.514171    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 12:17:16.514299 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.514183    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-p752x\""
Sep 09 12:17:16.514965 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.514951    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:17:16.522355 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.522332    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"]
Sep 09 12:17:16.547138 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.547112    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f0e857b-2645-4746-8d73-e0b10d799af9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fpgd6\" (UID: \"5f0e857b-2645-4746-8d73-e0b10d799af9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"
Sep 09 12:17:16.547260 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.547158    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k49zb\" (UniqueName: \"kubernetes.io/projected/5f0e857b-2645-4746-8d73-e0b10d799af9-kube-api-access-k49zb\") pod \"openshift-lws-operator-fd8ccff4c-fpgd6\" (UID: \"5f0e857b-2645-4746-8d73-e0b10d799af9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"
Sep 09 12:17:16.648395 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.648357    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f0e857b-2645-4746-8d73-e0b10d799af9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fpgd6\" (UID: \"5f0e857b-2645-4746-8d73-e0b10d799af9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"
Sep 09 12:17:16.648543 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.648416    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k49zb\" (UniqueName: \"kubernetes.io/projected/5f0e857b-2645-4746-8d73-e0b10d799af9-kube-api-access-k49zb\") pod \"openshift-lws-operator-fd8ccff4c-fpgd6\" (UID: \"5f0e857b-2645-4746-8d73-e0b10d799af9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"
Sep 09 12:17:16.648780 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.648749    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f0e857b-2645-4746-8d73-e0b10d799af9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fpgd6\" (UID: \"5f0e857b-2645-4746-8d73-e0b10d799af9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"
Sep 09 12:17:16.658908 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.658886    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k49zb\" (UniqueName: \"kubernetes.io/projected/5f0e857b-2645-4746-8d73-e0b10d799af9-kube-api-access-k49zb\") pod \"openshift-lws-operator-fd8ccff4c-fpgd6\" (UID: \"5f0e857b-2645-4746-8d73-e0b10d799af9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"
Sep 09 12:17:16.820949 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:16.820872    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"
Sep 09 12:17:17.095562 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:17.095448    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6"]
Sep 09 12:17:17.097802 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:17:17.097772    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f0e857b_2645_4746_8d73_e0b10d799af9.slice/crio-57eb7ce1e2d111ad13c867122244a7007699d678134c055b50968820ab26ed85 WatchSource:0}: Error finding container 57eb7ce1e2d111ad13c867122244a7007699d678134c055b50968820ab26ed85: Status 404 returned error can't find the container with id 57eb7ce1e2d111ad13c867122244a7007699d678134c055b50968820ab26ed85
Sep 09 12:17:17.099587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:17.099568    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:17:17.477405 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:17.477323    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6" event={"ID":"5f0e857b-2645-4746-8d73-e0b10d799af9","Type":"ContainerStarted","Data":"57eb7ce1e2d111ad13c867122244a7007699d678134c055b50968820ab26ed85"}
Sep 09 12:17:20.490209 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:20.490167    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6" event={"ID":"5f0e857b-2645-4746-8d73-e0b10d799af9","Type":"ContainerStarted","Data":"1298f23a872f09b1febd0e70df1680b657f79a8ece97b8640cce9474f68093cb"}
Sep 09 12:17:20.508950 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:20.508906    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fpgd6" podStartSLOduration=2.100378246 podStartE2EDuration="4.508891914s" totalImagesPullingTime="2.408513668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:17:20.506912905 +0000 UTC m=+385.148532134" watchObservedRunningTime="2026-09-09 12:17:20.508891914 +0000 UTC m=+385.150511118"
Sep 09 12:17:40.045400 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.045357    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"]
Sep 09 12:17:40.056564 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.056534    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.059760 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.059729    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 12:17:40.059904 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.059833    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 12:17:40.060587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.060569    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 12:17:40.061135 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.061117    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 12:17:40.061248 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.061226    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ssn29\""
Sep 09 12:17:40.069981 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.069961    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"]
Sep 09 12:17:40.157501 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.157463    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1de04984-4073-43bc-bd71-33e8156d5527-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.157652 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.157530    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1de04984-4073-43bc-bd71-33e8156d5527-manifests\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.157652 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.157565    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1de04984-4073-43bc-bd71-33e8156d5527-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.157652 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.157607    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1de04984-4073-43bc-bd71-33e8156d5527-tmp\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.157652 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.157636    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b598t\" (UniqueName: \"kubernetes.io/projected/1de04984-4073-43bc-bd71-33e8156d5527-kube-api-access-b598t\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.258546 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.258509    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1de04984-4073-43bc-bd71-33e8156d5527-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.258731 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.258557    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1de04984-4073-43bc-bd71-33e8156d5527-manifests\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.258731 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.258588    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1de04984-4073-43bc-bd71-33e8156d5527-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.258813 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.258769    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1de04984-4073-43bc-bd71-33e8156d5527-tmp\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.258853 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.258822    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b598t\" (UniqueName: \"kubernetes.io/projected/1de04984-4073-43bc-bd71-33e8156d5527-kube-api-access-b598t\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.259011 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.258994    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1de04984-4073-43bc-bd71-33e8156d5527-manifests\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.261254 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.261223    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1de04984-4073-43bc-bd71-33e8156d5527-tmp\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.261379 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.261361    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1de04984-4073-43bc-bd71-33e8156d5527-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.261441 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.261425    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1de04984-4073-43bc-bd71-33e8156d5527-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.267938 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.267910    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b598t\" (UniqueName: \"kubernetes.io/projected/1de04984-4073-43bc-bd71-33e8156d5527-kube-api-access-b598t\") pod \"opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk\" (UID: \"1de04984-4073-43bc-bd71-33e8156d5527\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.367303 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.367219    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:40.648168 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:40.648084    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"]
Sep 09 12:17:40.650751 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:17:40.650721    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de04984_4073_43bc_bd71_33e8156d5527.slice/crio-76e105b553f67b1cbcfc42d419a67f92e986e258cc72267e61aa8fa94649c90c WatchSource:0}: Error finding container 76e105b553f67b1cbcfc42d419a67f92e986e258cc72267e61aa8fa94649c90c: Status 404 returned error can't find the container with id 76e105b553f67b1cbcfc42d419a67f92e986e258cc72267e61aa8fa94649c90c
Sep 09 12:17:41.580306 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:41.580260    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk" event={"ID":"1de04984-4073-43bc-bd71-33e8156d5527","Type":"ContainerStarted","Data":"76e105b553f67b1cbcfc42d419a67f92e986e258cc72267e61aa8fa94649c90c"}
Sep 09 12:17:44.598921 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:44.598878    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk" event={"ID":"1de04984-4073-43bc-bd71-33e8156d5527","Type":"ContainerStarted","Data":"c01a55cda9805a1bca8cc88292372af0eea4f9f24da586bd0768df4beca7981a"}
Sep 09 12:17:45.606087 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:45.606053    2585 generic.go:356] "Generic (PLEG): container finished" podID="1de04984-4073-43bc-bd71-33e8156d5527" containerID="c01a55cda9805a1bca8cc88292372af0eea4f9f24da586bd0768df4beca7981a" exitCode=0
Sep 09 12:17:45.606465 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:45.606093    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk" event={"ID":"1de04984-4073-43bc-bd71-33e8156d5527","Type":"ContainerDied","Data":"c01a55cda9805a1bca8cc88292372af0eea4f9f24da586bd0768df4beca7981a"}
Sep 09 12:17:46.612187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:46.612152    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk" event={"ID":"1de04984-4073-43bc-bd71-33e8156d5527","Type":"ContainerStarted","Data":"a44c93d503d1957e02b210734fad9f24c1528bb5c9c73a66ab6618e35aca7ea4"}
Sep 09 12:17:46.612609 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:46.612310    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:17:46.640900 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:46.640851    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk" podStartSLOduration=2.116354559 podStartE2EDuration="6.640834948s" totalImagesPullingTime="3.520932321s" totalInitContainerRuntime="1.003548068s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:17:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:17:46.640432705 +0000 UTC m=+411.282051912" watchObservedRunningTime="2026-09-09 12:17:46.640834948 +0000 UTC m=+411.282454152"
Sep 09 12:17:57.619578 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:17:57.619545    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk"
Sep 09 12:18:12.326896 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.326861    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"]
Sep 09 12:18:12.335857 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.335829    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.338014 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.337993    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 12:18:12.338140 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.338020    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-jdkj6\""
Sep 09 12:18:12.338258 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.338242    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 12:18:12.338325 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.338243    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 12:18:12.341902 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.341872    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"]
Sep 09 12:18:12.438300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.438264    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.438498 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.438328    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.438498 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.438358    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.438498 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.438428    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9279200-2465-4063-ae1f-59d2116f018a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.438498 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.438475    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9279200-2465-4063-ae1f-59d2116f018a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.438737 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.438627    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.438737 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.438684    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbq5f\" (UniqueName: \"kubernetes.io/projected/d9279200-2465-4063-ae1f-59d2116f018a-kube-api-access-cbq5f\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.438824 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.438738    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9279200-2465-4063-ae1f-59d2116f018a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.438824 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.438769    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.539310 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539260    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9279200-2465-4063-ae1f-59d2116f018a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.539310 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539312    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9279200-2465-4063-ae1f-59d2116f018a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.539625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539356    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.539625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539377    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbq5f\" (UniqueName: \"kubernetes.io/projected/d9279200-2465-4063-ae1f-59d2116f018a-kube-api-access-cbq5f\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.539625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539401    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9279200-2465-4063-ae1f-59d2116f018a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.539625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539432    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.539625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539471    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.539625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539525    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.539625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539557    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.540058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539851    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.540058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.539877    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.540151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.540074    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.540151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.540095    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.540390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.540374    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9279200-2465-4063-ae1f-59d2116f018a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.541970 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.541947    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9279200-2465-4063-ae1f-59d2116f018a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.542187 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.542168    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9279200-2465-4063-ae1f-59d2116f018a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.549248 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.549223    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbq5f\" (UniqueName: \"kubernetes.io/projected/d9279200-2465-4063-ae1f-59d2116f018a-kube-api-access-cbq5f\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.549730 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.549685    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9279200-2465-4063-ae1f-59d2116f018a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp\" (UID: \"d9279200-2465-4063-ae1f-59d2116f018a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.648485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.648404    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:12.927674 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:12.927647    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"]
Sep 09 12:18:12.929974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:18:12.929931    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9279200_2465_4063_ae1f_59d2116f018a.slice/crio-6c192281f701e4782e1a3736c553b1b6411ef0bf748824d196f3ad7c0b35e060 WatchSource:0}: Error finding container 6c192281f701e4782e1a3736c553b1b6411ef0bf748824d196f3ad7c0b35e060: Status 404 returned error can't find the container with id 6c192281f701e4782e1a3736c553b1b6411ef0bf748824d196f3ad7c0b35e060
Sep 09 12:18:13.729962 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:13.729930    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp" event={"ID":"d9279200-2465-4063-ae1f-59d2116f018a","Type":"ContainerStarted","Data":"6c192281f701e4782e1a3736c553b1b6411ef0bf748824d196f3ad7c0b35e060"}
Sep 09 12:18:15.731640 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:15.731604    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:18:15.731942 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:15.731677    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:18:15.731942 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:15.731721    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:18:16.745424 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:16.745340    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp" event={"ID":"d9279200-2465-4063-ae1f-59d2116f018a","Type":"ContainerStarted","Data":"64d4fbfe59354875d7855556611a2f3048bded52a0cb95e8f01568c555038734"}
Sep 09 12:18:16.774096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:16.774020    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp" podStartSLOduration=1.974580815 podStartE2EDuration="4.774004558s" totalImagesPullingTime="2.799423743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:18:16.772037682 +0000 UTC m=+441.413656888" watchObservedRunningTime="2026-09-09 12:18:16.774004558 +0000 UTC m=+441.415623761"
Sep 09 12:18:17.649467 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:17.649433    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:17.654014 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:17.653986    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:17.750125 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:17.750094    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:17.751278 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:17.751256    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp"
Sep 09 12:18:19.223882 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.223850    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"]
Sep 09 12:18:19.227164 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.227147    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.231684 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.231657    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8qz6n\""
Sep 09 12:18:19.231819 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.231665    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 09 12:18:19.231819 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.231724    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 09 12:18:19.231819 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.231738    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 09 12:18:19.248599 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.248572    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"]
Sep 09 12:18:19.300025 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.299993    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6wb9\" (UniqueName: \"kubernetes.io/projected/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-kube-api-access-b6wb9\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.300195 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.300065    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-metrics-cert\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.300195 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.300117    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-manager-config\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.300195 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.300133    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-cert\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.401356 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.401318    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-manager-config\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.401356 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.401358    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-cert\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.401573 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.401382    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6wb9\" (UniqueName: \"kubernetes.io/projected/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-kube-api-access-b6wb9\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.401573 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.401436    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-metrics-cert\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.402017 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.401993    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-manager-config\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.403964 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.403933    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-cert\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.404085 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.403984    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-metrics-cert\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.415796 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.415774    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6wb9\" (UniqueName: \"kubernetes.io/projected/8902ddbd-b2ae-4dc7-b360-1664f7d2203f-kube-api-access-b6wb9\") pod \"lws-controller-manager-df65498f9-sgpss\" (UID: \"8902ddbd-b2ae-4dc7-b360-1664f7d2203f\") " pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.536468 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.536437    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:19.831683 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:19.831661    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"]
Sep 09 12:18:19.833573 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:18:19.833551    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8902ddbd_b2ae_4dc7_b360_1664f7d2203f.slice/crio-c4e17d0bc4fd1b11b7ff63e290c1d959f25b2253988007b9659c721e006846e7 WatchSource:0}: Error finding container c4e17d0bc4fd1b11b7ff63e290c1d959f25b2253988007b9659c721e006846e7: Status 404 returned error can't find the container with id c4e17d0bc4fd1b11b7ff63e290c1d959f25b2253988007b9659c721e006846e7
Sep 09 12:18:20.774350 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:20.774244    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss" event={"ID":"8902ddbd-b2ae-4dc7-b360-1664f7d2203f","Type":"ContainerStarted","Data":"c4e17d0bc4fd1b11b7ff63e290c1d959f25b2253988007b9659c721e006846e7"}
Sep 09 12:18:21.784435 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:21.784290    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss" event={"ID":"8902ddbd-b2ae-4dc7-b360-1664f7d2203f","Type":"ContainerStarted","Data":"7a331c6770957432b1a8068e86b17279432c0c9232712f494dc6cad2fe9e702c"}
Sep 09 12:18:21.784948 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:21.784839    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:21.840821 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:21.840759    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss" podStartSLOduration=1.041103552 podStartE2EDuration="2.840736399s" totalImagesPullingTime="1.799632847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:18:21.837946345 +0000 UTC m=+446.479565548" watchObservedRunningTime="2026-09-09 12:18:21.840736399 +0000 UTC m=+446.482355604"
Sep 09 12:18:22.019842 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.019811    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-92v57"]
Sep 09 12:18:22.023631 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.023605    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57"
Sep 09 12:18:22.033219 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.033191    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 09 12:18:22.033952 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.033919    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-65fs8\""
Sep 09 12:18:22.065322 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.065292    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-92v57"]
Sep 09 12:18:22.127567 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.127532    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/325079e6-e247-4adc-b0e9-a37fae9d7b57-cert\") pod \"kserve-controller-manager-d6f4b65b8-92v57\" (UID: \"325079e6-e247-4adc-b0e9-a37fae9d7b57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57"
Sep 09 12:18:22.127764 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.127632    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xfq5\" (UniqueName: \"kubernetes.io/projected/325079e6-e247-4adc-b0e9-a37fae9d7b57-kube-api-access-5xfq5\") pod \"kserve-controller-manager-d6f4b65b8-92v57\" (UID: \"325079e6-e247-4adc-b0e9-a37fae9d7b57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57"
Sep 09 12:18:22.228716 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.228668    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xfq5\" (UniqueName: \"kubernetes.io/projected/325079e6-e247-4adc-b0e9-a37fae9d7b57-kube-api-access-5xfq5\") pod \"kserve-controller-manager-d6f4b65b8-92v57\" (UID: \"325079e6-e247-4adc-b0e9-a37fae9d7b57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57"
Sep 09 12:18:22.228883 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.228796    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/325079e6-e247-4adc-b0e9-a37fae9d7b57-cert\") pod \"kserve-controller-manager-d6f4b65b8-92v57\" (UID: \"325079e6-e247-4adc-b0e9-a37fae9d7b57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57"
Sep 09 12:18:22.231380 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.231357    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/325079e6-e247-4adc-b0e9-a37fae9d7b57-cert\") pod \"kserve-controller-manager-d6f4b65b8-92v57\" (UID: \"325079e6-e247-4adc-b0e9-a37fae9d7b57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57"
Sep 09 12:18:22.260305 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.260274    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xfq5\" (UniqueName: \"kubernetes.io/projected/325079e6-e247-4adc-b0e9-a37fae9d7b57-kube-api-access-5xfq5\") pod \"kserve-controller-manager-d6f4b65b8-92v57\" (UID: \"325079e6-e247-4adc-b0e9-a37fae9d7b57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57"
Sep 09 12:18:22.336399 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.336305    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57"
Sep 09 12:18:22.633325 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.633197    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-92v57"]
Sep 09 12:18:22.635148 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:18:22.635118    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325079e6_e247_4adc_b0e9_a37fae9d7b57.slice/crio-19c583b70d0a8790e9f29d8ea8c8a342a87d12056f9646f0e66101aac4ac03b4 WatchSource:0}: Error finding container 19c583b70d0a8790e9f29d8ea8c8a342a87d12056f9646f0e66101aac4ac03b4: Status 404 returned error can't find the container with id 19c583b70d0a8790e9f29d8ea8c8a342a87d12056f9646f0e66101aac4ac03b4
Sep 09 12:18:22.790589 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:22.790559    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57" event={"ID":"325079e6-e247-4adc-b0e9-a37fae9d7b57","Type":"ContainerStarted","Data":"19c583b70d0a8790e9f29d8ea8c8a342a87d12056f9646f0e66101aac4ac03b4"}
Sep 09 12:18:26.812277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:26.812239    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57" event={"ID":"325079e6-e247-4adc-b0e9-a37fae9d7b57","Type":"ContainerStarted","Data":"a8cdd7636eed51ab7472d03f8a469abae46787b4fb6b7326e32977c0d4b885db"}
Sep 09 12:18:26.812673 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:26.812378    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57"
Sep 09 12:18:26.844038 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:26.843981    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57" podStartSLOduration=2.313767452 podStartE2EDuration="5.843960544s" totalImagesPullingTime="3.530193092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:18:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:18:26.843210576 +0000 UTC m=+451.484829779" watchObservedRunningTime="2026-09-09 12:18:26.843960544 +0000 UTC m=+451.485579749"
Sep 09 12:18:32.792304 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:32.792272    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-df65498f9-sgpss"
Sep 09 12:18:57.822056 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:18:57.822018    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92v57"
Sep 09 12:19:25.889328 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:25.889290    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz"]
Sep 09 12:19:25.892761 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:25.892743    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz"
Sep 09 12:19:25.895222 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:25.895194    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-8bj89\""
Sep 09 12:19:25.895345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:25.895194    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 12:19:25.895345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:25.895327    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 09 12:19:25.895956 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:25.895935    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 12:19:25.905217 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:25.905195    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz"]
Sep 09 12:19:25.990148 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:25.990120    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqksq\" (UniqueName: \"kubernetes.io/projected/1cc91038-30ea-44f5-b002-0fb514b7eba6-kube-api-access-rqksq\") pod \"dns-operator-controller-manager-7b84cd7c9c-97mrz\" (UID: \"1cc91038-30ea-44f5-b002-0fb514b7eba6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz"
Sep 09 12:19:26.091399 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:26.091374    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqksq\" (UniqueName: \"kubernetes.io/projected/1cc91038-30ea-44f5-b002-0fb514b7eba6-kube-api-access-rqksq\") pod \"dns-operator-controller-manager-7b84cd7c9c-97mrz\" (UID: \"1cc91038-30ea-44f5-b002-0fb514b7eba6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz"
Sep 09 12:19:26.104028 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:26.104000    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqksq\" (UniqueName: \"kubernetes.io/projected/1cc91038-30ea-44f5-b002-0fb514b7eba6-kube-api-access-rqksq\") pod \"dns-operator-controller-manager-7b84cd7c9c-97mrz\" (UID: \"1cc91038-30ea-44f5-b002-0fb514b7eba6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz"
Sep 09 12:19:26.204223 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:26.204135    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz"
Sep 09 12:19:26.482448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:26.482336    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz"]
Sep 09 12:19:26.485181 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:19:26.485157    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc91038_30ea_44f5_b002_0fb514b7eba6.slice/crio-b7967144cb0d41629217967ef512697b8fc34a34c88dc58f5b569d59051c6bc1 WatchSource:0}: Error finding container b7967144cb0d41629217967ef512697b8fc34a34c88dc58f5b569d59051c6bc1: Status 404 returned error can't find the container with id b7967144cb0d41629217967ef512697b8fc34a34c88dc58f5b569d59051c6bc1
Sep 09 12:19:27.077682 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:27.077649    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz" event={"ID":"1cc91038-30ea-44f5-b002-0fb514b7eba6","Type":"ContainerStarted","Data":"b7967144cb0d41629217967ef512697b8fc34a34c88dc58f5b569d59051c6bc1"}
Sep 09 12:19:28.630288 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:28.630248    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-x6mkb"]
Sep 09 12:19:28.639024 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:28.638996    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-x6mkb"
Sep 09 12:19:28.642339 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:28.642315    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-v2mrf\""
Sep 09 12:19:28.648660 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:28.648637    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-x6mkb"]
Sep 09 12:19:28.818172 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:28.818140    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v87st\" (UniqueName: \"kubernetes.io/projected/c54e1303-c5a5-40cc-ad25-43165a3cb13e-kube-api-access-v87st\") pod \"authorino-operator-76b478bd78-x6mkb\" (UID: \"c54e1303-c5a5-40cc-ad25-43165a3cb13e\") " pod="kuadrant-system/authorino-operator-76b478bd78-x6mkb"
Sep 09 12:19:28.919315 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:28.919230    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v87st\" (UniqueName: \"kubernetes.io/projected/c54e1303-c5a5-40cc-ad25-43165a3cb13e-kube-api-access-v87st\") pod \"authorino-operator-76b478bd78-x6mkb\" (UID: \"c54e1303-c5a5-40cc-ad25-43165a3cb13e\") " pod="kuadrant-system/authorino-operator-76b478bd78-x6mkb"
Sep 09 12:19:28.929402 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:28.929372    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v87st\" (UniqueName: \"kubernetes.io/projected/c54e1303-c5a5-40cc-ad25-43165a3cb13e-kube-api-access-v87st\") pod \"authorino-operator-76b478bd78-x6mkb\" (UID: \"c54e1303-c5a5-40cc-ad25-43165a3cb13e\") " pod="kuadrant-system/authorino-operator-76b478bd78-x6mkb"
Sep 09 12:19:28.953210 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:28.953178    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-x6mkb"
Sep 09 12:19:29.421281 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:29.421248    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-x6mkb"]
Sep 09 12:19:29.430449 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:19:29.430416    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54e1303_c5a5_40cc_ad25_43165a3cb13e.slice/crio-80ec7723d921adbeb13f18c1c6940e658863edd38673787d2c57bf88bcbee9eb WatchSource:0}: Error finding container 80ec7723d921adbeb13f18c1c6940e658863edd38673787d2c57bf88bcbee9eb: Status 404 returned error can't find the container with id 80ec7723d921adbeb13f18c1c6940e658863edd38673787d2c57bf88bcbee9eb
Sep 09 12:19:30.096538 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:30.096502    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz" event={"ID":"1cc91038-30ea-44f5-b002-0fb514b7eba6","Type":"ContainerStarted","Data":"d68d8d004d38558633607f84bc17f25a4635031d704370effc4a6f2821a939ae"}
Sep 09 12:19:30.096995 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:30.096664    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz"
Sep 09 12:19:30.097877 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:30.097856    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-x6mkb" event={"ID":"c54e1303-c5a5-40cc-ad25-43165a3cb13e","Type":"ContainerStarted","Data":"80ec7723d921adbeb13f18c1c6940e658863edd38673787d2c57bf88bcbee9eb"}
Sep 09 12:19:30.127930 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:30.127884    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz" podStartSLOduration=1.719203463 podStartE2EDuration="5.127868233s" totalImagesPullingTime="3.40866477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:19:30.126099182 +0000 UTC m=+514.767718415" watchObservedRunningTime="2026-09-09 12:19:30.127868233 +0000 UTC m=+514.769487436"
Sep 09 12:19:31.104681 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:31.104646    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-x6mkb" event={"ID":"c54e1303-c5a5-40cc-ad25-43165a3cb13e","Type":"ContainerStarted","Data":"5d459a944d15ed46a7ac4eb69c0c95199fecd36ff6326936e34f0c0612435b78"}
Sep 09 12:19:31.105106 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:31.104828    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-x6mkb"
Sep 09 12:19:34.831579 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:34.831525    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-x6mkb" podStartSLOduration=5.351711012 podStartE2EDuration="6.831507189s" totalImagesPullingTime="1.479796177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:19:31.138026968 +0000 UTC m=+515.779646173" watchObservedRunningTime="2026-09-09 12:19:34.831507189 +0000 UTC m=+519.473126392"
Sep 09 12:19:34.832723 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:34.832688    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"]
Sep 09 12:19:34.836202 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:34.836187    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:34.841117 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:34.841099    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-lfxf6\""
Sep 09 12:19:34.846806 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:34.846785    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"]
Sep 09 12:19:34.880716 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:34.880661    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwvj2\" (UniqueName: \"kubernetes.io/projected/ce60d65e-a789-47ce-a32c-c740b0b64e76-kube-api-access-vwvj2\") pod \"limitador-operator-controller-manager-74495c69f8-zphdk\" (UID: \"ce60d65e-a789-47ce-a32c-c740b0b64e76\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:34.981744 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:34.981687    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwvj2\" (UniqueName: \"kubernetes.io/projected/ce60d65e-a789-47ce-a32c-c740b0b64e76-kube-api-access-vwvj2\") pod \"limitador-operator-controller-manager-74495c69f8-zphdk\" (UID: \"ce60d65e-a789-47ce-a32c-c740b0b64e76\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:34.995848 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:34.995816    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwvj2\" (UniqueName: \"kubernetes.io/projected/ce60d65e-a789-47ce-a32c-c740b0b64e76-kube-api-access-vwvj2\") pod \"limitador-operator-controller-manager-74495c69f8-zphdk\" (UID: \"ce60d65e-a789-47ce-a32c-c740b0b64e76\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:35.147027 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:35.146944    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:35.425294 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:35.425268    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"]
Sep 09 12:19:35.427727 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:19:35.427684    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce60d65e_a789_47ce_a32c_c740b0b64e76.slice/crio-cc34281f54b13054fea0d6edf1621c7e2ba1a3eed43d451b88fa08890f559ddb WatchSource:0}: Error finding container cc34281f54b13054fea0d6edf1621c7e2ba1a3eed43d451b88fa08890f559ddb: Status 404 returned error can't find the container with id cc34281f54b13054fea0d6edf1621c7e2ba1a3eed43d451b88fa08890f559ddb
Sep 09 12:19:36.129191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:36.129157    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk" event={"ID":"ce60d65e-a789-47ce-a32c-c740b0b64e76","Type":"ContainerStarted","Data":"cc34281f54b13054fea0d6edf1621c7e2ba1a3eed43d451b88fa08890f559ddb"}
Sep 09 12:19:37.135332 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:37.135293    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk" event={"ID":"ce60d65e-a789-47ce-a32c-c740b0b64e76","Type":"ContainerStarted","Data":"af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36"}
Sep 09 12:19:37.135735 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:37.135437    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:37.153504 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:37.153458    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk" podStartSLOduration=1.6043389380000002 podStartE2EDuration="3.153442359s" totalImagesPullingTime="1.549103421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:19:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:19:37.151066856 +0000 UTC m=+521.792686063" watchObservedRunningTime="2026-09-09 12:19:37.153442359 +0000 UTC m=+521.795061563"
Sep 09 12:19:41.107247 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:41.107218    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-97mrz"
Sep 09 12:19:42.112873 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:42.112843    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-x6mkb"
Sep 09 12:19:48.142357 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:48.142276    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:51.207885 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.207845    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"]
Sep 09 12:19:51.212488 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.212454    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:51.217510 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.217485    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mc7gz\""
Sep 09 12:19:51.229154 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.229130    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"]
Sep 09 12:19:51.335548 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.335512    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w48cw\" (UniqueName: \"kubernetes.io/projected/d150ae8a-533d-4d60-a0e8-89dca0cec636-kube-api-access-w48cw\") pod \"kuadrant-operator-controller-manager-5f685c496f-grltd\" (UID: \"d150ae8a-533d-4d60-a0e8-89dca0cec636\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:51.335750 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.335558    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d150ae8a-533d-4d60-a0e8-89dca0cec636-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-grltd\" (UID: \"d150ae8a-533d-4d60-a0e8-89dca0cec636\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:51.436392 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.436347    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w48cw\" (UniqueName: \"kubernetes.io/projected/d150ae8a-533d-4d60-a0e8-89dca0cec636-kube-api-access-w48cw\") pod \"kuadrant-operator-controller-manager-5f685c496f-grltd\" (UID: \"d150ae8a-533d-4d60-a0e8-89dca0cec636\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:51.436550 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.436408    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d150ae8a-533d-4d60-a0e8-89dca0cec636-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-grltd\" (UID: \"d150ae8a-533d-4d60-a0e8-89dca0cec636\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:51.436841 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.436823    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d150ae8a-533d-4d60-a0e8-89dca0cec636-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-grltd\" (UID: \"d150ae8a-533d-4d60-a0e8-89dca0cec636\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:51.455763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.455727    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w48cw\" (UniqueName: \"kubernetes.io/projected/d150ae8a-533d-4d60-a0e8-89dca0cec636-kube-api-access-w48cw\") pod \"kuadrant-operator-controller-manager-5f685c496f-grltd\" (UID: \"d150ae8a-533d-4d60-a0e8-89dca0cec636\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:51.523723 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.523672    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:51.813079 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:51.813053    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"]
Sep 09 12:19:51.813730 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:19:51.813689    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd150ae8a_533d_4d60_a0e8_89dca0cec636.slice/crio-6a90e4e604b95674325cca2d2368003762c1d08e943af51268c669f03072cbc5 WatchSource:0}: Error finding container 6a90e4e604b95674325cca2d2368003762c1d08e943af51268c669f03072cbc5: Status 404 returned error can't find the container with id 6a90e4e604b95674325cca2d2368003762c1d08e943af51268c669f03072cbc5
Sep 09 12:19:52.117934 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.117859    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"]
Sep 09 12:19:52.129257 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.129228    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"]
Sep 09 12:19:52.199645 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.199613    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"]
Sep 09 12:19:52.199900 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.199854    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk" podUID="ce60d65e-a789-47ce-a32c-c740b0b64e76" containerName="manager" containerID="cri-o://af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36" gracePeriod=2
Sep 09 12:19:52.209470 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.209389    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"]
Sep 09 12:19:52.212131 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:52.211931    2585 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ce60d65e-a789-47ce-a32c-c740b0b64e76\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-zphdk\": pods \"limitador-operator-controller-manager-74495c69f8-zphdk\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:52.227076 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.227053    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd"]
Sep 09 12:19:52.227418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.227407    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce60d65e-a789-47ce-a32c-c740b0b64e76" containerName="manager"
Sep 09 12:19:52.227466 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.227419    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce60d65e-a789-47ce-a32c-c740b0b64e76" containerName="manager"
Sep 09 12:19:52.227503 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.227484    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce60d65e-a789-47ce-a32c-c740b0b64e76" containerName="manager"
Sep 09 12:19:52.247197 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.247159    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd"]
Sep 09 12:19:52.247328 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.247276    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd"
Sep 09 12:19:52.249608 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:52.249576    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zphdk\" is forbidden: User \"system:node:ip-10-0-131-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-207.ec2.internal' and this object" podUID="ce60d65e-a789-47ce-a32c-c740b0b64e76" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:52.345174 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.345139    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh69c\" (UniqueName: \"kubernetes.io/projected/4aae3551-7e56-455c-aadc-d58e489cb63a-kube-api-access-zh69c\") pod \"limitador-operator-controller-manager-74495c69f8-g5fwd\" (UID: \"4aae3551-7e56-455c-aadc-d58e489cb63a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd"
Sep 09 12:19:52.445606 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.445573    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh69c\" (UniqueName: \"kubernetes.io/projected/4aae3551-7e56-455c-aadc-d58e489cb63a-kube-api-access-zh69c\") pod \"limitador-operator-controller-manager-74495c69f8-g5fwd\" (UID: \"4aae3551-7e56-455c-aadc-d58e489cb63a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd"
Sep 09 12:19:52.457175 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.457141    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh69c\" (UniqueName: \"kubernetes.io/projected/4aae3551-7e56-455c-aadc-d58e489cb63a-kube-api-access-zh69c\") pod \"limitador-operator-controller-manager-74495c69f8-g5fwd\" (UID: \"4aae3551-7e56-455c-aadc-d58e489cb63a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd"
Sep 09 12:19:52.460736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.460692    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:52.462866 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:52.462750    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zphdk\" is forbidden: User \"system:node:ip-10-0-131-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-207.ec2.internal' and this object" podUID="ce60d65e-a789-47ce-a32c-c740b0b64e76" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:52.546933 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.546902    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwvj2\" (UniqueName: \"kubernetes.io/projected/ce60d65e-a789-47ce-a32c-c740b0b64e76-kube-api-access-vwvj2\") pod \"ce60d65e-a789-47ce-a32c-c740b0b64e76\" (UID: \"ce60d65e-a789-47ce-a32c-c740b0b64e76\") "
Sep 09 12:19:52.549155 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.549134    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce60d65e-a789-47ce-a32c-c740b0b64e76-kube-api-access-vwvj2" (OuterVolumeSpecName: "kube-api-access-vwvj2") pod "ce60d65e-a789-47ce-a32c-c740b0b64e76" (UID: "ce60d65e-a789-47ce-a32c-c740b0b64e76"). InnerVolumeSpecName "kube-api-access-vwvj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:19:52.577463 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.577434    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd"
Sep 09 12:19:52.648249 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.648082    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwvj2\" (UniqueName: \"kubernetes.io/projected/ce60d65e-a789-47ce-a32c-c740b0b64e76-kube-api-access-vwvj2\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:19:52.897639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:52.897568    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd"]
Sep 09 12:19:52.900956 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:19:52.900929    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aae3551_7e56_455c_aadc_d58e489cb63a.slice/crio-233bfc84430f626044796d7db86b7ac88fe781be166250432dab43651f2e7a0c WatchSource:0}: Error finding container 233bfc84430f626044796d7db86b7ac88fe781be166250432dab43651f2e7a0c: Status 404 returned error can't find the container with id 233bfc84430f626044796d7db86b7ac88fe781be166250432dab43651f2e7a0c
Sep 09 12:19:53.217571 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:53.217481    2585 generic.go:356] "Generic (PLEG): container finished" podID="ce60d65e-a789-47ce-a32c-c740b0b64e76" containerID="af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36" exitCode=0
Sep 09 12:19:53.217571 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:53.217531    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:53.218140 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:53.217590    2585 scope.go:117] "RemoveContainer" containerID="af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36"
Sep 09 12:19:53.219854 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:53.219808    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd" event={"ID":"4aae3551-7e56-455c-aadc-d58e489cb63a","Type":"ContainerStarted","Data":"8de93d1b0b0b530e31f375dad1cf835b005f156fea21a20ea9e4ed4e13cd25d5"}
Sep 09 12:19:53.219854 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:53.219828    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zphdk\" is forbidden: User \"system:node:ip-10-0-131-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-207.ec2.internal' and this object" podUID="ce60d65e-a789-47ce-a32c-c740b0b64e76" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:53.220043 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:53.219878    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd" event={"ID":"4aae3551-7e56-455c-aadc-d58e489cb63a","Type":"ContainerStarted","Data":"233bfc84430f626044796d7db86b7ac88fe781be166250432dab43651f2e7a0c"}
Sep 09 12:19:53.220117 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:53.220035    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd"
Sep 09 12:19:53.222754 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:53.222728    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zphdk\" is forbidden: User \"system:node:ip-10-0-131-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-207.ec2.internal' and this object" podUID="ce60d65e-a789-47ce-a32c-c740b0b64e76" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:53.228827 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:53.228809    2585 scope.go:117] "RemoveContainer" containerID="af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36"
Sep 09 12:19:53.229140 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:53.229119    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36\": container with ID starting with af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36 not found: ID does not exist" containerID="af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36"
Sep 09 12:19:53.229195 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:53.229153    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36"} err="failed to get container status \"af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36\": rpc error: code = NotFound desc = could not find container \"af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36\": container with ID starting with af690eca5f0b5cbff24a4c1a27869062794c9658affc3e0225d3ac03ecd67e36 not found: ID does not exist"
Sep 09 12:19:53.243090 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:53.243039    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd" podStartSLOduration=1.24302393 podStartE2EDuration="1.24302393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:19:53.241572129 +0000 UTC m=+537.883191343" watchObservedRunningTime="2026-09-09 12:19:53.24302393 +0000 UTC m=+537.884643141"
Sep 09 12:19:53.245346 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:53.245313    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zphdk\" is forbidden: User \"system:node:ip-10-0-131-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-207.ec2.internal' and this object" podUID="ce60d65e-a789-47ce-a32c-c740b0b64e76" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zphdk"
Sep 09 12:19:53.900627 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:53.900591    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce60d65e-a789-47ce-a32c-c740b0b64e76" path="/var/lib/kubelet/pods/ce60d65e-a789-47ce-a32c-c740b0b64e76/volumes"
Sep 09 12:19:56.239534 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:56.239496    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd" podUID="d150ae8a-533d-4d60-a0e8-89dca0cec636" containerName="manager" containerID="cri-o://59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f" gracePeriod=2
Sep 09 12:19:56.241404 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:56.241368    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-grltd\" is forbidden: User \"system:node:ip-10-0-131-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-207.ec2.internal' and this object" podUID="d150ae8a-533d-4d60-a0e8-89dca0cec636" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:56.482151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:56.482127    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:56.484125 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:56.484094    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-grltd\" is forbidden: User \"system:node:ip-10-0-131-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-207.ec2.internal' and this object" podUID="d150ae8a-533d-4d60-a0e8-89dca0cec636" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:56.584105 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:56.584080    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d150ae8a-533d-4d60-a0e8-89dca0cec636-extensions-socket-volume\") pod \"d150ae8a-533d-4d60-a0e8-89dca0cec636\" (UID: \"d150ae8a-533d-4d60-a0e8-89dca0cec636\") "
Sep 09 12:19:56.584259 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:56.584140    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w48cw\" (UniqueName: \"kubernetes.io/projected/d150ae8a-533d-4d60-a0e8-89dca0cec636-kube-api-access-w48cw\") pod \"d150ae8a-533d-4d60-a0e8-89dca0cec636\" (UID: \"d150ae8a-533d-4d60-a0e8-89dca0cec636\") "
Sep 09 12:19:56.584351 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:56.584330    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d150ae8a-533d-4d60-a0e8-89dca0cec636-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d150ae8a-533d-4d60-a0e8-89dca0cec636" (UID: "d150ae8a-533d-4d60-a0e8-89dca0cec636"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:19:56.584445 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:56.584433    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d150ae8a-533d-4d60-a0e8-89dca0cec636-extensions-socket-volume\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:19:56.586287 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:56.586268    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d150ae8a-533d-4d60-a0e8-89dca0cec636-kube-api-access-w48cw" (OuterVolumeSpecName: "kube-api-access-w48cw") pod "d150ae8a-533d-4d60-a0e8-89dca0cec636" (UID: "d150ae8a-533d-4d60-a0e8-89dca0cec636"). InnerVolumeSpecName "kube-api-access-w48cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:19:56.685632 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:56.685601    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w48cw\" (UniqueName: \"kubernetes.io/projected/d150ae8a-533d-4d60-a0e8-89dca0cec636-kube-api-access-w48cw\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:19:57.245557 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:57.245524    2585 generic.go:356] "Generic (PLEG): container finished" podID="d150ae8a-533d-4d60-a0e8-89dca0cec636" containerID="59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f" exitCode=2
Sep 09 12:19:57.246064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:57.245576    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:57.246064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:57.245628    2585 scope.go:117] "RemoveContainer" containerID="59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f"
Sep 09 12:19:57.247678 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:57.247652    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-grltd\" is forbidden: User \"system:node:ip-10-0-131-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-207.ec2.internal' and this object" podUID="d150ae8a-533d-4d60-a0e8-89dca0cec636" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:57.254505 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:57.254491    2585 scope.go:117] "RemoveContainer" containerID="59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f"
Sep 09 12:19:57.254800 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:57.254783    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f\": container with ID starting with 59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f not found: ID does not exist" containerID="59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f"
Sep 09 12:19:57.254850 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:57.254810    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f"} err="failed to get container status \"59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f\": rpc error: code = NotFound desc = could not find container \"59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f\": container with ID starting with 59b0a684ec454a9f06eb19ebabc948f0b60b582556ba50630dcdca7611e6d99f not found: ID does not exist"
Sep 09 12:19:57.255578 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:19:57.255563    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-grltd\" is forbidden: User \"system:node:ip-10-0-131-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-207.ec2.internal' and this object" podUID="d150ae8a-533d-4d60-a0e8-89dca0cec636" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-grltd"
Sep 09 12:19:57.900130 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:19:57.900089    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d150ae8a-533d-4d60-a0e8-89dca0cec636" path="/var/lib/kubelet/pods/d150ae8a-533d-4d60-a0e8-89dca0cec636/volumes"
Sep 09 12:20:04.229007 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:04.228974    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-g5fwd"
Sep 09 12:20:05.110552 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.110519    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"]
Sep 09 12:20:05.110900 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.110889    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d150ae8a-533d-4d60-a0e8-89dca0cec636" containerName="manager"
Sep 09 12:20:05.110948 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.110902    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d150ae8a-533d-4d60-a0e8-89dca0cec636" containerName="manager"
Sep 09 12:20:05.110981 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.110967    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d150ae8a-533d-4d60-a0e8-89dca0cec636" containerName="manager"
Sep 09 12:20:05.115347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.115315    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:20:05.118091 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.118072    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mc7gz\""
Sep 09 12:20:05.134139 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.134109    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"]
Sep 09 12:20:05.260080 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.260038    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjfzt\" (UniqueName: \"kubernetes.io/projected/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-kube-api-access-rjfzt\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sphfn\" (UID: \"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:20:05.260536 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.260174    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sphfn\" (UID: \"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:20:05.361161 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.361065    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjfzt\" (UniqueName: \"kubernetes.io/projected/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-kube-api-access-rjfzt\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sphfn\" (UID: \"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:20:05.361320 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.361161    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sphfn\" (UID: \"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:20:05.361556 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.361537    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sphfn\" (UID: \"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:20:05.375931 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.375906    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjfzt\" (UniqueName: \"kubernetes.io/projected/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-kube-api-access-rjfzt\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sphfn\" (UID: \"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:20:05.425879 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.425843    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:20:05.714803 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:05.714780    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"]
Sep 09 12:20:05.717246 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:20:05.717220    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda363b8_fb6b_4e7b_b7ef_91b4c2315bbc.slice/crio-7a0ebc01abfb4517871e6b82040f6b567ecec31016ecae2ad4b4131c2f79cbbf WatchSource:0}: Error finding container 7a0ebc01abfb4517871e6b82040f6b567ecec31016ecae2ad4b4131c2f79cbbf: Status 404 returned error can't find the container with id 7a0ebc01abfb4517871e6b82040f6b567ecec31016ecae2ad4b4131c2f79cbbf
Sep 09 12:20:06.289893 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:06.289860    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn" event={"ID":"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc","Type":"ContainerStarted","Data":"f1a1c09ab5f3b63b295a17cb1b855235abd6770aaf3c412783dd43c6eb4d09d9"}
Sep 09 12:20:06.290299 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:06.289938    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn" event={"ID":"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc","Type":"ContainerStarted","Data":"7a0ebc01abfb4517871e6b82040f6b567ecec31016ecae2ad4b4131c2f79cbbf"}
Sep 09 12:20:06.290299 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:06.289997    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:20:06.314519 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:06.314464    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn" podStartSLOduration=1.314446512 podStartE2EDuration="1.314446512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:06.313080271 +0000 UTC m=+550.954699476" watchObservedRunningTime="2026-09-09 12:20:06.314446512 +0000 UTC m=+550.956065769"
Sep 09 12:20:17.297302 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:20:17.297261    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:21:51.273420 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.273381    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-chp7l"]
Sep 09 12:21:51.277302 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.277281    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l"
Sep 09 12:21:51.279801 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.279779    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-w889h\""
Sep 09 12:21:51.279924 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.279784    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 12:21:51.287570 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.287546    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-chp7l"]
Sep 09 12:21:51.317150 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.317116    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b6e2ff7-9e23-49c1-aa14-b29047aeef39-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-chp7l\" (UID: \"7b6e2ff7-9e23-49c1-aa14-b29047aeef39\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l"
Sep 09 12:21:51.317338 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.317242    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z75rj\" (UniqueName: \"kubernetes.io/projected/7b6e2ff7-9e23-49c1-aa14-b29047aeef39-kube-api-access-z75rj\") pod \"payload-processing-5b54b7fbb7-chp7l\" (UID: \"7b6e2ff7-9e23-49c1-aa14-b29047aeef39\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l"
Sep 09 12:21:51.417829 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.417789    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b6e2ff7-9e23-49c1-aa14-b29047aeef39-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-chp7l\" (UID: \"7b6e2ff7-9e23-49c1-aa14-b29047aeef39\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l"
Sep 09 12:21:51.418010 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.417897    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z75rj\" (UniqueName: \"kubernetes.io/projected/7b6e2ff7-9e23-49c1-aa14-b29047aeef39-kube-api-access-z75rj\") pod \"payload-processing-5b54b7fbb7-chp7l\" (UID: \"7b6e2ff7-9e23-49c1-aa14-b29047aeef39\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l"
Sep 09 12:21:51.418496 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.418476    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b6e2ff7-9e23-49c1-aa14-b29047aeef39-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-chp7l\" (UID: \"7b6e2ff7-9e23-49c1-aa14-b29047aeef39\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l"
Sep 09 12:21:51.426419 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.426398    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z75rj\" (UniqueName: \"kubernetes.io/projected/7b6e2ff7-9e23-49c1-aa14-b29047aeef39-kube-api-access-z75rj\") pod \"payload-processing-5b54b7fbb7-chp7l\" (UID: \"7b6e2ff7-9e23-49c1-aa14-b29047aeef39\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l"
Sep 09 12:21:51.589610 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.589527    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l"
Sep 09 12:21:51.861104 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:51.861073    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-chp7l"]
Sep 09 12:21:51.863969 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:21:51.863938    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b6e2ff7_9e23_49c1_aa14_b29047aeef39.slice/crio-c739af8659bb4661b9d90ccbb06e6d8de74deb41e287769ec6b0df239fed4760 WatchSource:0}: Error finding container c739af8659bb4661b9d90ccbb06e6d8de74deb41e287769ec6b0df239fed4760: Status 404 returned error can't find the container with id c739af8659bb4661b9d90ccbb06e6d8de74deb41e287769ec6b0df239fed4760
Sep 09 12:21:52.810034 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:52.809987    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l" event={"ID":"7b6e2ff7-9e23-49c1-aa14-b29047aeef39","Type":"ContainerStarted","Data":"c739af8659bb4661b9d90ccbb06e6d8de74deb41e287769ec6b0df239fed4760"}
Sep 09 12:21:54.823888 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:54.823851    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l" event={"ID":"7b6e2ff7-9e23-49c1-aa14-b29047aeef39","Type":"ContainerStarted","Data":"bd69e698b982807d0e7a4241b3103d80b734d17885f0d8954b57627a68a2518a"}
Sep 09 12:21:54.824372 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:54.823962    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l"
Sep 09 12:21:54.841798 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:21:54.841756    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l" podStartSLOduration=1.470022278 podStartE2EDuration="3.841740881s" totalImagesPullingTime="2.371718603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:21:54.839925946 +0000 UTC m=+659.481545155" watchObservedRunningTime="2026-09-09 12:21:54.841740881 +0000 UTC m=+659.483360082"
Sep 09 12:22:05.831862 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:05.831827    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-chp7l"
Sep 09 12:22:12.494994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.494961    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"]
Sep 09 12:22:12.499000 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.498980    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.501550 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.501530    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 12:22:12.502006 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.501975    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 09 12:22:12.502131 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.502083    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 12:22:12.503982 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.502513    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6shtk\""
Sep 09 12:22:12.509782 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.509751    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"]
Sep 09 12:22:12.613447 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.613410    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.613447 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.613449    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/575ca116-ef49-422e-b08d-4538a4c11491-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.613688 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.613476    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.613688 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.613574    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.613688 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.613629    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.613688 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.613687    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqfmm\" (UniqueName: \"kubernetes.io/projected/575ca116-ef49-422e-b08d-4538a4c11491-kube-api-access-qqfmm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.715163 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.715119    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqfmm\" (UniqueName: \"kubernetes.io/projected/575ca116-ef49-422e-b08d-4538a4c11491-kube-api-access-qqfmm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.715335 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.715180    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.715335 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.715231    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/575ca116-ef49-422e-b08d-4538a4c11491-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.715335 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.715281    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.715335 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.715334    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.715483 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.715367    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.715618 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.715599    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.715687 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.715666    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.715813 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.715797    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.717774 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.717750    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/575ca116-ef49-422e-b08d-4538a4c11491-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.717996 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.717980    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/575ca116-ef49-422e-b08d-4538a4c11491-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.724190 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.724167    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqfmm\" (UniqueName: \"kubernetes.io/projected/575ca116-ef49-422e-b08d-4538a4c11491-kube-api-access-qqfmm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j6gzd\" (UID: \"575ca116-ef49-422e-b08d-4538a4c11491\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:12.816831 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:12.816790    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:13.090289 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:13.090265    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"]
Sep 09 12:22:13.092109 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:22:13.092071    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575ca116_ef49_422e_b08d_4538a4c11491.slice/crio-de842f6ed88eab3f686cd70d20841be3d64060bc13ddb75591e6f2e8baa77864 WatchSource:0}: Error finding container de842f6ed88eab3f686cd70d20841be3d64060bc13ddb75591e6f2e8baa77864: Status 404 returned error can't find the container with id de842f6ed88eab3f686cd70d20841be3d64060bc13ddb75591e6f2e8baa77864
Sep 09 12:22:13.919179 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:13.919139    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd" event={"ID":"575ca116-ef49-422e-b08d-4538a4c11491","Type":"ContainerStarted","Data":"de842f6ed88eab3f686cd70d20841be3d64060bc13ddb75591e6f2e8baa77864"}
Sep 09 12:22:14.925734 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:14.925677    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd" event={"ID":"575ca116-ef49-422e-b08d-4538a4c11491","Type":"ContainerStarted","Data":"316ac9f97105d61f9d356dcc8aa1753870de381ef0aa69b9a772aea2996ea2b6"}
Sep 09 12:22:14.943466 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:14.943413    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd" podStartSLOduration=1.9123118529999998 podStartE2EDuration="2.943397397s" totalImagesPullingTime="1.031085544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:22:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:22:14.942182577 +0000 UTC m=+679.583801779" watchObservedRunningTime="2026-09-09 12:22:14.943397397 +0000 UTC m=+679.585016623"
Sep 09 12:22:22.816942 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:22.816906    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:22.817340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:22.816977    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:22.830099 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:22.830075    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:22.976997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:22.976970    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j6gzd"
Sep 09 12:22:57.785597 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:57.785561    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"]
Sep 09 12:22:57.789596 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:57.789532    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:57.792056 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:57.792025    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 09 12:22:57.798313 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:57.798288    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"]
Sep 09 12:22:57.940588 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:57.940555    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:57.940588 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:57.940594    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:57.940826 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:57.940615    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2zmw\" (UniqueName: \"kubernetes.io/projected/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-kube-api-access-c2zmw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:57.940826 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:57.940638    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:57.940826 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:57.940687    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:57.940826 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:57.940767    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.041687 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.041592    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.041687 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.041632    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.041687 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.041654    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2zmw\" (UniqueName: \"kubernetes.io/projected/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-kube-api-access-c2zmw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.041687 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.041673    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.042085 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.041824    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.042085 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.041937    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.042085 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.042059    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.042211 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.042143    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.042296 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.042279    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.044139 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.044117    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.044326 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.044307    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.058903 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.058879    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2zmw\" (UniqueName: \"kubernetes.io/projected/3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75-kube-api-access-c2zmw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x5q8w\" (UID: \"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.102248 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.102223    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:22:58.389039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.389012    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"]
Sep 09 12:22:58.391054 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:22:58.391023    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc61d5e_2f57_4861_a1fa_be6fbf5c8d75.slice/crio-cf2542c671c2ab56133259881e387194a46e44493eb104f49ad2f1d61561841e WatchSource:0}: Error finding container cf2542c671c2ab56133259881e387194a46e44493eb104f49ad2f1d61561841e: Status 404 returned error can't find the container with id cf2542c671c2ab56133259881e387194a46e44493eb104f49ad2f1d61561841e
Sep 09 12:22:58.392563 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:58.392547    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:22:59.145054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:59.145019    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w" event={"ID":"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75","Type":"ContainerStarted","Data":"803ec9eb4496ef51f2c3cb466960ea284d9333340d8af597c589e67daffa78e0"}
Sep 09 12:22:59.145527 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:59.145069    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w" event={"ID":"3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75","Type":"ContainerStarted","Data":"cf2542c671c2ab56133259881e387194a46e44493eb104f49ad2f1d61561841e"}
Sep 09 12:22:59.162793 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:22:59.162745    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w" podStartSLOduration=1.92606397 podStartE2EDuration="2.162729421s" totalImagesPullingTime="236.665451ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:22:59.161996904 +0000 UTC m=+723.803616108" watchObservedRunningTime="2026-09-09 12:22:59.162729421 +0000 UTC m=+723.804348625"
Sep 09 12:23:08.103215 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:23:08.103172    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:23:08.103614 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:23:08.103237    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:23:08.115768 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:23:08.115742    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:23:08.202084 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:23:08.202058    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x5q8w"
Sep 09 12:24:28.622039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.621820    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"]
Sep 09 12:24:28.625826 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.625806    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.628350 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.628310    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-c554db-openshift-default-dockercfg-mgzjt\""
Sep 09 12:24:28.639054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.639035    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"]
Sep 09 12:24:28.670194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.670151    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-socket\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.670391 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.670244    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-certs\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.670391 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.670302    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-data\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.670391 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.670358    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5hjt\" (UniqueName: \"kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-kube-api-access-d5hjt\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.670616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.670456    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-credential-socket\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.670616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.670486    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-envoy\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.670616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.670530    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-podinfo\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.670616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.670552    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-token\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.670616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.670594    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76595636-4e31-433e-86de-3c4df5e8ed8a-istiod-ca-cert\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.771708 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.771666    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-certs\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.771916 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.771734    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-data\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.771916 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.771775    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5hjt\" (UniqueName: \"kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-kube-api-access-d5hjt\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.771916 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.771848    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-credential-socket\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.771916 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.771873    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-envoy\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.771916 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.771899    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-podinfo\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.772193 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.771922    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-token\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.772193 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.771950    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76595636-4e31-433e-86de-3c4df5e8ed8a-istiod-ca-cert\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.772193 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.772026    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-socket\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.772193 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.772099    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-certs\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.772193 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.772150    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-data\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.772460 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.772302    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-socket\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.772584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.772558    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-credential-socket\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.772907 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.772878    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76595636-4e31-433e-86de-3c4df5e8ed8a-istiod-ca-cert\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.775362 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.775210    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-podinfo\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.775489 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.775430    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-envoy\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.796384 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.796294    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-token\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.796384 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.796325    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5hjt\" (UniqueName: \"kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-kube-api-access-d5hjt\") pod \"e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") " pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:28.937949 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:28.937866    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:29.221439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:29.221414    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"]
Sep 09 12:24:29.223891 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:24:29.223860    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76595636_4e31_433e_86de_3c4df5e8ed8a.slice/crio-cacb641841cb053627706628a4b8947b65fb0e11e14dd0ce80625842b4d7aec7 WatchSource:0}: Error finding container cacb641841cb053627706628a4b8947b65fb0e11e14dd0ce80625842b4d7aec7: Status 404 returned error can't find the container with id cacb641841cb053627706628a4b8947b65fb0e11e14dd0ce80625842b4d7aec7
Sep 09 12:24:29.226489 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:29.226455    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:29.226594 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:29.226534    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:29.226594 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:29.226580    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:29.593321 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:29.593287    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" event={"ID":"76595636-4e31-433e-86de-3c4df5e8ed8a","Type":"ContainerStarted","Data":"abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e"}
Sep 09 12:24:29.593503 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:29.593348    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" event={"ID":"76595636-4e31-433e-86de-3c4df5e8ed8a","Type":"ContainerStarted","Data":"cacb641841cb053627706628a4b8947b65fb0e11e14dd0ce80625842b4d7aec7"}
Sep 09 12:24:29.614169 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:29.614114    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" podStartSLOduration=1.6140950379999999 podStartE2EDuration="1.614095038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:24:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:24:29.613593912 +0000 UTC m=+814.255213144" watchObservedRunningTime="2026-09-09 12:24:29.614095038 +0000 UTC m=+814.255714244"
Sep 09 12:24:29.938855 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:29.938772    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:30.943923 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:30.943892    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:31.603290 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:31.603260    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:31.604145 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:31.604126    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:24:34.788743 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:34.788710    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"]
Sep 09 12:24:34.792617 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:34.792594    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:24:34.798252 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:34.798231    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-c554db\""
Sep 09 12:24:34.804096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:34.804068    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"]
Sep 09 12:24:34.827165 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:34.827129    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3453ea77-0da0-4e6a-a413-8ddb50448697-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl\" (UID: \"3453ea77-0da0-4e6a-a413-8ddb50448697\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:24:34.827333 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:34.827278    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvftd\" (UniqueName: \"kubernetes.io/projected/3453ea77-0da0-4e6a-a413-8ddb50448697-kube-api-access-zvftd\") pod \"payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl\" (UID: \"3453ea77-0da0-4e6a-a413-8ddb50448697\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:24:34.928101 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:34.928065    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvftd\" (UniqueName: \"kubernetes.io/projected/3453ea77-0da0-4e6a-a413-8ddb50448697-kube-api-access-zvftd\") pod \"payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl\" (UID: \"3453ea77-0da0-4e6a-a413-8ddb50448697\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:24:34.928260 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:34.928136    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3453ea77-0da0-4e6a-a413-8ddb50448697-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl\" (UID: \"3453ea77-0da0-4e6a-a413-8ddb50448697\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:24:34.928812 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:34.928794    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3453ea77-0da0-4e6a-a413-8ddb50448697-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl\" (UID: \"3453ea77-0da0-4e6a-a413-8ddb50448697\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:24:34.937748 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:34.937713    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvftd\" (UniqueName: \"kubernetes.io/projected/3453ea77-0da0-4e6a-a413-8ddb50448697-kube-api-access-zvftd\") pod \"payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl\" (UID: \"3453ea77-0da0-4e6a-a413-8ddb50448697\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:24:35.108038 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.107955    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:24:35.388066 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.387978    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"]
Sep 09 12:24:35.390401 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:24:35.390362    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3453ea77_0da0_4e6a_a413_8ddb50448697.slice/crio-531400f5da1a989143efcf7934c3b87b49e75caa0acaacfe4b151958a127e634 WatchSource:0}: Error finding container 531400f5da1a989143efcf7934c3b87b49e75caa0acaacfe4b151958a127e634: Status 404 returned error can't find the container with id 531400f5da1a989143efcf7934c3b87b49e75caa0acaacfe4b151958a127e634
Sep 09 12:24:35.626267 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.626229    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl" event={"ID":"3453ea77-0da0-4e6a-a413-8ddb50448697","Type":"ContainerStarted","Data":"4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea"}
Sep 09 12:24:35.626431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.626289    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl" event={"ID":"3453ea77-0da0-4e6a-a413-8ddb50448697","Type":"ContainerStarted","Data":"531400f5da1a989143efcf7934c3b87b49e75caa0acaacfe4b151958a127e634"}
Sep 09 12:24:35.645741 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.645620    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl" podStartSLOduration=1.6456020599999999 podStartE2EDuration="1.64560206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:24:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:24:35.645387374 +0000 UTC m=+820.287006579" watchObservedRunningTime="2026-09-09 12:24:35.64560206 +0000 UTC m=+820.287221264"
Sep 09 12:24:35.726179 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.726135    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"]
Sep 09 12:24:35.729720 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.729685    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:24:35.732038 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.732018    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-bc0270\""
Sep 09 12:24:35.739847 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.739819    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"]
Sep 09 12:24:35.836438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.836400    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx26s\" (UniqueName: \"kubernetes.io/projected/baceb9d6-9b27-4a4a-be78-1f4730f15109-kube-api-access-dx26s\") pod \"payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625\" (UID: \"baceb9d6-9b27-4a4a-be78-1f4730f15109\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:24:35.836438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.836435    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baceb9d6-9b27-4a4a-be78-1f4730f15109-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625\" (UID: \"baceb9d6-9b27-4a4a-be78-1f4730f15109\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:24:35.938029 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.937941    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx26s\" (UniqueName: \"kubernetes.io/projected/baceb9d6-9b27-4a4a-be78-1f4730f15109-kube-api-access-dx26s\") pod \"payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625\" (UID: \"baceb9d6-9b27-4a4a-be78-1f4730f15109\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:24:35.938029 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.937991    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baceb9d6-9b27-4a4a-be78-1f4730f15109-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625\" (UID: \"baceb9d6-9b27-4a4a-be78-1f4730f15109\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:24:35.938635 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.938617    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baceb9d6-9b27-4a4a-be78-1f4730f15109-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625\" (UID: \"baceb9d6-9b27-4a4a-be78-1f4730f15109\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:24:35.946780 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:35.946750    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx26s\" (UniqueName: \"kubernetes.io/projected/baceb9d6-9b27-4a4a-be78-1f4730f15109-kube-api-access-dx26s\") pod \"payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625\" (UID: \"baceb9d6-9b27-4a4a-be78-1f4730f15109\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:24:36.042307 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:36.042265    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:24:36.326992 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:36.326954    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"]
Sep 09 12:24:36.334189 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:24:36.334165    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaceb9d6_9b27_4a4a_be78_1f4730f15109.slice/crio-1cf940dd2d1fff888394851cf8bc3ba645cccdc9fd99959c47fdf3f426d524b4 WatchSource:0}: Error finding container 1cf940dd2d1fff888394851cf8bc3ba645cccdc9fd99959c47fdf3f426d524b4: Status 404 returned error can't find the container with id 1cf940dd2d1fff888394851cf8bc3ba645cccdc9fd99959c47fdf3f426d524b4
Sep 09 12:24:36.632508 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:36.632410    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625" event={"ID":"baceb9d6-9b27-4a4a-be78-1f4730f15109","Type":"ContainerStarted","Data":"df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959"}
Sep 09 12:24:36.632741 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:36.632693    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:24:36.632833 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:36.632767    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:24:36.632833 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:36.632791    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625" event={"ID":"baceb9d6-9b27-4a4a-be78-1f4730f15109","Type":"ContainerStarted","Data":"1cf940dd2d1fff888394851cf8bc3ba645cccdc9fd99959c47fdf3f426d524b4"}
Sep 09 12:24:36.649434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:36.649362    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625" podStartSLOduration=1.649342595 podStartE2EDuration="1.649342595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:24:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:24:36.648914811 +0000 UTC m=+821.290534017" watchObservedRunningTime="2026-09-09 12:24:36.649342595 +0000 UTC m=+821.290961800"
Sep 09 12:24:38.620898 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.620868    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"]
Sep 09 12:24:38.624633 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.624610    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.627210 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.627177    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c801b865-openshift-default-dockercfg-dgcf6\""
Sep 09 12:24:38.636669 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.636643    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"]
Sep 09 12:24:38.669230 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.669191    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-token\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.669409 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.669251    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-podinfo\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.669409 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.669290    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-socket\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.669409 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.669342    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-certs\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.669409 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.669366    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-data\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.669409 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.669399    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2h75\" (UniqueName: \"kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-kube-api-access-x2h75\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.669661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.669429    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-credential-socket\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.669661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.669476    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b126680-71c8-4809-af0c-f2b0a0840b92-istiod-ca-cert\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.669661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.669525    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-envoy\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770303 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770262    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-token\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770303 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770310    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-podinfo\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770333    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-socket\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770379    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-certs\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770403    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-data\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770437    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2h75\" (UniqueName: \"kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-kube-api-access-x2h75\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770460    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-credential-socket\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770507    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b126680-71c8-4809-af0c-f2b0a0840b92-istiod-ca-cert\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770895 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770575    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-envoy\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770895 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770838    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-certs\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.770895 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770873    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-socket\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.771052 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.770932    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-data\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.771733 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.771290    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-credential-socket\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.771851 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.771793    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b126680-71c8-4809-af0c-f2b0a0840b92-istiod-ca-cert\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.773731 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.773681    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-envoy\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.773989 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.773970    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-podinfo\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.779420 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.779394    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-token\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.779984 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.779965    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2h75\" (UniqueName: \"kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-kube-api-access-x2h75\") pod \"e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") " pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:38.935969 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:38.935872    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:39.221138 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.221111    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"]
Sep 09 12:24:39.222643 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:24:39.222614    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b126680_71c8_4809_af0c_f2b0a0840b92.slice/crio-fc05b558bc38de163ab62a2e88677fe9118bb85b1b02f793d8f37ba1e4e39135 WatchSource:0}: Error finding container fc05b558bc38de163ab62a2e88677fe9118bb85b1b02f793d8f37ba1e4e39135: Status 404 returned error can't find the container with id fc05b558bc38de163ab62a2e88677fe9118bb85b1b02f793d8f37ba1e4e39135
Sep 09 12:24:39.225174 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.225143    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:39.225269 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.225205    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:39.225269 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.225231    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:39.325888 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.322375    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"]
Sep 09 12:24:39.326543 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.326524    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.329081 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.329059    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-a1e1f1-openshift-default-dockercfg-77fn2\""
Sep 09 12:24:39.339476 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.339388    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"]
Sep 09 12:24:39.376506 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.376476    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-credential-socket\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.376727 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.376525    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istiod-ca-cert\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.376727 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.376555    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-socket\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.376727 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.376614    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-envoy\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.376727 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.376671    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-certs\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.376727 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.376691    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-token\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.377001 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.376736    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-podinfo\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.377001 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.376757    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-data\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.377001 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.376776    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n74cg\" (UniqueName: \"kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-kube-api-access-n74cg\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.478515 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.478420    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-envoy\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.478678 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.478536    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-certs\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.478868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.478740    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-token\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.478868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.478779    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-podinfo\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.478868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.478815    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-data\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.478868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.478849    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n74cg\" (UniqueName: \"kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-kube-api-access-n74cg\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.479136 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.478937    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-credential-socket\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.479136 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.478968    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-certs\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.479136 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.478980    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istiod-ca-cert\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.479136 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.479019    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-socket\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.479351 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.479321    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-data\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.479351 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.479340    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-socket\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.479686 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.479575    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-credential-socket\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.479686 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.479608    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istiod-ca-cert\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.481810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.481785    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-envoy\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.481908 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.481896    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-podinfo\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.488888 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.488861    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-token\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.489126 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.489107    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n74cg\" (UniqueName: \"kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-kube-api-access-n74cg\") pod \"e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") " pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.643399 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.643368    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:39.651772 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.651740    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" event={"ID":"2b126680-71c8-4809-af0c-f2b0a0840b92","Type":"ContainerStarted","Data":"78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73"}
Sep 09 12:24:39.651894 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.651804    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" event={"ID":"2b126680-71c8-4809-af0c-f2b0a0840b92","Type":"ContainerStarted","Data":"fc05b558bc38de163ab62a2e88677fe9118bb85b1b02f793d8f37ba1e4e39135"}
Sep 09 12:24:39.675766 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.675675    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" podStartSLOduration=1.675657014 podStartE2EDuration="1.675657014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:24:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:24:39.67462858 +0000 UTC m=+824.316247788" watchObservedRunningTime="2026-09-09 12:24:39.675657014 +0000 UTC m=+824.317276218"
Sep 09 12:24:39.930066 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.929974    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"]
Sep 09 12:24:39.932817 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:24:39.932777    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a9e9f9_eba9_4d4c_99ab_3b7525bf27e1.slice/crio-29cb9503406c9d53897b4818db3ba455afd5e952867e626a59ded6dc5fb4bec5 WatchSource:0}: Error finding container 29cb9503406c9d53897b4818db3ba455afd5e952867e626a59ded6dc5fb4bec5: Status 404 returned error can't find the container with id 29cb9503406c9d53897b4818db3ba455afd5e952867e626a59ded6dc5fb4bec5
Sep 09 12:24:39.935129 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.935101    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:39.935224 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.935174    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:39.935224 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.935214    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:39.935997 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:39.935978    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:40.660737 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:40.660673    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" event={"ID":"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1","Type":"ContainerStarted","Data":"a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573"}
Sep 09 12:24:40.661248 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:40.660801    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" event={"ID":"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1","Type":"ContainerStarted","Data":"29cb9503406c9d53897b4818db3ba455afd5e952867e626a59ded6dc5fb4bec5"}
Sep 09 12:24:40.687507 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:40.687452    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podStartSLOduration=1.6874342850000001 podStartE2EDuration="1.687434285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:24:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:24:40.684524626 +0000 UTC m=+825.326143835" watchObservedRunningTime="2026-09-09 12:24:40.687434285 +0000 UTC m=+825.329053489"
Sep 09 12:24:40.941722 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:40.941616    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:41.644501 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:41.644463    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:41.649686 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:41.649655    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:41.666038 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:41.666010    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:41.666462 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:41.666062    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:41.666808 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:41.666786    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:41.666897 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:41.666841    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:24:47.639761 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:47.639732    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:24:48.646863 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:48.646835    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:24:57.383738 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.368146    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"]
Sep 09 12:24:57.394691 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.394648    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"]
Sep 09 12:24:57.394930 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.394911    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.397736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.397680    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-2a142a-openshift-default-dockercfg-tlk5w\""
Sep 09 12:24:57.496438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.496400    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-socket\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.496615 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.496457    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-credential-socket\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.496615 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.496524    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d480be69-2c10-4eeb-9819-56ac690fb8c4-istiod-ca-cert\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.496615 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.496557    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-podinfo\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.496615 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.496603    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-data\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.496811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.496639    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-envoy\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.496811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.496678    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-token\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.496811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.496729    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-certs\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.496811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.496751    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcpcm\" (UniqueName: \"kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-kube-api-access-rcpcm\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.597780 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.597738    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-certs\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.597780 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.597790    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcpcm\" (UniqueName: \"kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-kube-api-access-rcpcm\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.598040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.597837    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-socket\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.598040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.597885    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-credential-socket\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.598040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.597934    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d480be69-2c10-4eeb-9819-56ac690fb8c4-istiod-ca-cert\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.598040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.597972    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-podinfo\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.598040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.598018    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-data\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.598278 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.598071    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-envoy\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.598278 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.598107    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-token\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.598278 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.598238    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-certs\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.598420 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.598332    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-socket\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.598776 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.598737    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-data\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.599185 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.599144    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-credential-socket\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.599304 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.599277    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d480be69-2c10-4eeb-9819-56ac690fb8c4-istiod-ca-cert\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.601268 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.601249    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-envoy\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.602435 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.602413    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-podinfo\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.608034 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.608009    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-token\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.610324 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.610300    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcpcm\" (UniqueName: \"kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-kube-api-access-rcpcm\") pod \"e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") " pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.707686 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.707591    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:57.707887 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.707858    2585 patch_prober.go:28] interesting pod/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 12:24:57.707954 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.707918    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:24:57.710828 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.710807    2585 patch_prober.go:28] interesting pod/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 12:24:57.710924 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:57.710843    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:24:58.031153 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.031122    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"]
Sep 09 12:24:58.033417 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:24:58.033390    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd480be69_2c10_4eeb_9819_56ac690fb8c4.slice/crio-a7084b75fceac73756d46f64f05d6d38be2ba9552d44bf48172bf7bdb431cd07 WatchSource:0}: Error finding container a7084b75fceac73756d46f64f05d6d38be2ba9552d44bf48172bf7bdb431cd07: Status 404 returned error can't find the container with id a7084b75fceac73756d46f64f05d6d38be2ba9552d44bf48172bf7bdb431cd07
Sep 09 12:24:58.036102 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.036070    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:58.036209 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.036145    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:58.036209 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.036177    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:58.783749 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.783689    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" event={"ID":"d480be69-2c10-4eeb-9819-56ac690fb8c4","Type":"ContainerStarted","Data":"06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1"}
Sep 09 12:24:58.784272 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.783773    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" event={"ID":"d480be69-2c10-4eeb-9819-56ac690fb8c4","Type":"ContainerStarted","Data":"a7084b75fceac73756d46f64f05d6d38be2ba9552d44bf48172bf7bdb431cd07"}
Sep 09 12:24:58.786152 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.786125    2585 generic.go:356] "Generic (PLEG): container finished" podID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerID="78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73" exitCode=0
Sep 09 12:24:58.786311 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.786181    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" event={"ID":"2b126680-71c8-4809-af0c-f2b0a0840b92","Type":"ContainerDied","Data":"78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73"}
Sep 09 12:24:58.786731 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.786713    2585 scope.go:117] "RemoveContainer" containerID="78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73"
Sep 09 12:24:58.788246 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.788207    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:58.788368 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.788297    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:58.788368 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.788351    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:24:58.818349 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:58.818303    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" podStartSLOduration=1.8182856250000001 podStartE2EDuration="1.818285625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:24:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:24:58.816042271 +0000 UTC m=+843.457661476" watchObservedRunningTime="2026-09-09 12:24:58.818285625 +0000 UTC m=+843.459904830"
Sep 09 12:24:59.708491 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:59.708409    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:59.716710 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:59.716634    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:59.798305 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:59.797874    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" event={"ID":"2b126680-71c8-4809-af0c-f2b0a0840b92","Type":"ContainerStarted","Data":"ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600"}
Sep 09 12:24:59.798860 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:59.798838    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:59.803663 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:59.803606    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:24:59.937062 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:59.937036    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:24:59.939146 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:59.937358    2585 patch_prober.go:28] interesting pod/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 12:24:59.939146 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:24:59.937415    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 12:25:00.810572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:00.810533    2585 generic.go:356] "Generic (PLEG): container finished" podID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerID="a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573" exitCode=0
Sep 09 12:25:00.811100 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:00.810596    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" event={"ID":"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1","Type":"ContainerDied","Data":"a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573"}
Sep 09 12:25:00.811173 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:00.811159    2585 scope.go:117] "RemoveContainer" containerID="a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573"
Sep 09 12:25:00.812278 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:00.812251    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:25:00.812382 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:00.812315    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:25:00.812382 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:00.812361    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:25:00.937199 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:00.937158    2585 patch_prober.go:28] interesting pod/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 12:25:00.937323 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:00.937221    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 12:25:01.818802 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:01.818767    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" event={"ID":"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1","Type":"ContainerStarted","Data":"65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f"}
Sep 09 12:25:01.937391 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:01.937351    2585 patch_prober.go:28] interesting pod/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 12:25:01.937644 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:01.937424    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 12:25:02.574724 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.574613    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"]
Sep 09 12:25:02.579285 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.579255    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:25:02.584046 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.584016    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-2a142a\""
Sep 09 12:25:02.588778 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.588750    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"]
Sep 09 12:25:02.643899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.643859    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:25:02.644125 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.644100    2585 patch_prober.go:28] interesting pod/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 09 12:25:02.644209 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.644159    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 09 12:25:02.661734 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.661508    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a229715f-540f-4e60-9af2-8cb6ed1a0c37-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f\" (UID: \"a229715f-540f-4e60-9af2-8cb6ed1a0c37\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:25:02.661734 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.661624    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5gpp\" (UniqueName: \"kubernetes.io/projected/a229715f-540f-4e60-9af2-8cb6ed1a0c37-kube-api-access-h5gpp\") pod \"payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f\" (UID: \"a229715f-540f-4e60-9af2-8cb6ed1a0c37\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:25:02.764110 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.762633    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a229715f-540f-4e60-9af2-8cb6ed1a0c37-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f\" (UID: \"a229715f-540f-4e60-9af2-8cb6ed1a0c37\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:25:02.764110 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.762786    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5gpp\" (UniqueName: \"kubernetes.io/projected/a229715f-540f-4e60-9af2-8cb6ed1a0c37-kube-api-access-h5gpp\") pod \"payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f\" (UID: \"a229715f-540f-4e60-9af2-8cb6ed1a0c37\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:25:02.764110 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.764057    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a229715f-540f-4e60-9af2-8cb6ed1a0c37-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f\" (UID: \"a229715f-540f-4e60-9af2-8cb6ed1a0c37\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:25:02.773448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.773386    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5gpp\" (UniqueName: \"kubernetes.io/projected/a229715f-540f-4e60-9af2-8cb6ed1a0c37-kube-api-access-h5gpp\") pod \"payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f\" (UID: \"a229715f-540f-4e60-9af2-8cb6ed1a0c37\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:25:02.895194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.895099    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:25:02.937451 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.937396    2585 patch_prober.go:28] interesting pod/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 12:25:02.937615 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:02.937489    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 12:25:03.236478 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:03.236371    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"]
Sep 09 12:25:03.240271 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:25:03.240244    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda229715f_540f_4e60_9af2_8cb6ed1a0c37.slice/crio-c0e6045eda63467ac697577b358a4b43a51e47872f5dab4b433de8a197342f78 WatchSource:0}: Error finding container c0e6045eda63467ac697577b358a4b43a51e47872f5dab4b433de8a197342f78: Status 404 returned error can't find the container with id c0e6045eda63467ac697577b358a4b43a51e47872f5dab4b433de8a197342f78
Sep 09 12:25:03.644581 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:03.644508    2585 patch_prober.go:28] interesting pod/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 09 12:25:03.644581 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:03.644564    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 09 12:25:03.831940 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:03.831899    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f" event={"ID":"a229715f-540f-4e60-9af2-8cb6ed1a0c37","Type":"ContainerStarted","Data":"2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a"}
Sep 09 12:25:03.832151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:03.831986    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f" event={"ID":"a229715f-540f-4e60-9af2-8cb6ed1a0c37","Type":"ContainerStarted","Data":"c0e6045eda63467ac697577b358a4b43a51e47872f5dab4b433de8a197342f78"}
Sep 09 12:25:03.832151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:03.832010    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:25:03.851277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:03.851196    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f" podStartSLOduration=1.851162202 podStartE2EDuration="1.851162202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:03.848585278 +0000 UTC m=+848.490204484" watchObservedRunningTime="2026-09-09 12:25:03.851162202 +0000 UTC m=+848.492781412"
Sep 09 12:25:03.937636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:03.937238    2585 patch_prober.go:28] interesting pod/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 12:25:03.937636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:03.937291    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 12:25:04.644451 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:04.644420    2585 patch_prober.go:28] interesting pod/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 09 12:25:04.644715 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:04.644678    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 09 12:25:04.936803 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:04.936677    2585 patch_prober.go:28] interesting pod/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 12:25:04.936803 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:04.936774    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 12:25:05.644656 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:05.644615    2585 patch_prober.go:28] interesting pod/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 09 12:25:05.645072 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:05.644684    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 09 12:25:05.941211 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:05.941125    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:25:05.941527 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:05.941478    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:25:05.942509 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:05.942488    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:25:06.644567 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.644524    2585 patch_prober.go:28] interesting pod/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 09 12:25:06.646681 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.644605    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 09 12:25:06.745849 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.745815    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"]
Sep 09 12:25:06.749237 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.749203    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"
Sep 09 12:25:06.751864 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.751600    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:25:06.751864 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.751638    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:25:06.767259 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.767151    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"]
Sep 09 12:25:06.834839 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.834751    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d41482-3638-4d28-bef1-e2abd6027e44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c801b865-9dfzk\" (UID: \"11d41482-3638-4d28-bef1-e2abd6027e44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"
Sep 09 12:25:06.879071 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.879036    2585 generic.go:356] "Generic (PLEG): container finished" podID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerID="abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e" exitCode=0
Sep 09 12:25:06.879270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.879127    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" event={"ID":"76595636-4e31-433e-86de-3c4df5e8ed8a","Type":"ContainerDied","Data":"abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e"}
Sep 09 12:25:06.879786 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.879761    2585 scope.go:117] "RemoveContainer" containerID="abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e"
Sep 09 12:25:06.880970 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.880942    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:25:06.881034 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.881012    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:25:06.881071 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.881041    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:25:06.936286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.936258    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d41482-3638-4d28-bef1-e2abd6027e44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c801b865-9dfzk\" (UID: \"11d41482-3638-4d28-bef1-e2abd6027e44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"
Sep 09 12:25:06.936976 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:06.936953    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d41482-3638-4d28-bef1-e2abd6027e44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c801b865-9dfzk\" (UID: \"11d41482-3638-4d28-bef1-e2abd6027e44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"
Sep 09 12:25:07.080733 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:07.080674    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"
Sep 09 12:25:07.386361 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:07.386335    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"]
Sep 09 12:25:07.388517 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:25:07.388479    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d41482_3638_4d28_bef1_e2abd6027e44.slice/crio-e7013e4962e9f3f1afa2e900a91ad633cc77c4b82c7b5c64f1260069debd292b WatchSource:0}: Error finding container e7013e4962e9f3f1afa2e900a91ad633cc77c4b82c7b5c64f1260069debd292b: Status 404 returned error can't find the container with id e7013e4962e9f3f1afa2e900a91ad633cc77c4b82c7b5c64f1260069debd292b
Sep 09 12:25:07.644503 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:07.644414    2585 patch_prober.go:28] interesting pod/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 09 12:25:07.644503 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:07.644473    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 09 12:25:07.886438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:07.886396    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk" event={"ID":"11d41482-3638-4d28-bef1-e2abd6027e44","Type":"ContainerStarted","Data":"e7013e4962e9f3f1afa2e900a91ad633cc77c4b82c7b5c64f1260069debd292b"}
Sep 09 12:25:07.888485 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:07.888460    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" event={"ID":"76595636-4e31-433e-86de-3c4df5e8ed8a","Type":"ContainerStarted","Data":"4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c"}
Sep 09 12:25:07.938428 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:07.938337    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:25:07.938611 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:07.938586    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:25:07.938669 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:07.938642    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 09 12:25:08.644031 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:08.643988    2585 patch_prober.go:28] interesting pod/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 09 12:25:08.644218 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:08.644051    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 09 12:25:08.938644 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:08.938564    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:25:08.938644 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:08.938622    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 09 12:25:09.648160 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:09.648124    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:25:09.648432 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:09.648403    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:25:09.662728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:09.659311    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:25:09.939039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:09.938961    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:25:09.939039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:09.939016    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 09 12:25:10.938569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:10.938535    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:25:10.938770 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:10.938589    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 09 12:25:11.915632 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:11.915598    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk" event={"ID":"11d41482-3638-4d28-bef1-e2abd6027e44","Type":"ContainerStarted","Data":"aa79d6dc1cea207e5a201eab333696cc6ee7aa03dff2965c92a26136a879084e"}
Sep 09 12:25:11.935953 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:11.935901    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk" podStartSLOduration=2.289415948 podStartE2EDuration="5.935884641s" totalImagesPullingTime="3.646468693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:11.934956849 +0000 UTC m=+856.576576053" watchObservedRunningTime="2026-09-09 12:25:11.935884641 +0000 UTC m=+856.577503844"
Sep 09 12:25:11.938508 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:11.938488    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:25:11.938583 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:11.938531    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 09 12:25:12.922899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:12.922855    2585 generic.go:356] "Generic (PLEG): container finished" podID="11d41482-3638-4d28-bef1-e2abd6027e44" containerID="aa79d6dc1cea207e5a201eab333696cc6ee7aa03dff2965c92a26136a879084e" exitCode=0
Sep 09 12:25:12.923296 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:12.922946    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk" event={"ID":"11d41482-3638-4d28-bef1-e2abd6027e44","Type":"ContainerDied","Data":"aa79d6dc1cea207e5a201eab333696cc6ee7aa03dff2965c92a26136a879084e"}
Sep 09 12:25:12.938260 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:12.938222    2585 patch_prober.go:28] interesting pod/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:25:12.938403 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:12.938272    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 09 12:25:13.942583 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:13.942554    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:25:13.943026 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:13.943010    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:25:13.943660 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:13.943641    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:25:14.077799 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:14.077759    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"
Sep 09 12:25:14.228967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:14.228876    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d41482-3638-4d28-bef1-e2abd6027e44-maas-api-service-ca\") pod \"11d41482-3638-4d28-bef1-e2abd6027e44\" (UID: \"11d41482-3638-4d28-bef1-e2abd6027e44\") "
Sep 09 12:25:14.229221 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:14.229197    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d41482-3638-4d28-bef1-e2abd6027e44-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "11d41482-3638-4d28-bef1-e2abd6027e44" (UID: "11d41482-3638-4d28-bef1-e2abd6027e44"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:25:14.330113 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:14.330057    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d41482-3638-4d28-bef1-e2abd6027e44-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:14.839222 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:14.839187    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:25:14.936052 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:14.936019    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"
Sep 09 12:25:14.936189 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:14.936067    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk" event={"ID":"11d41482-3638-4d28-bef1-e2abd6027e44","Type":"ContainerDied","Data":"e7013e4962e9f3f1afa2e900a91ad633cc77c4b82c7b5c64f1260069debd292b"}
Sep 09 12:25:14.936189 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:14.936117    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7013e4962e9f3f1afa2e900a91ad633cc77c4b82c7b5c64f1260069debd292b"
Sep 09 12:25:24.419945 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:24.419907    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"]
Sep 09 12:25:24.423148 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:24.423122    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c801b865-9dfzk"]
Sep 09 12:25:24.571215 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:24.571177    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"]
Sep 09 12:25:24.571735 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:24.571682    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy" containerID="cri-o://ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600" gracePeriod=30
Sep 09 12:25:25.901197 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:25.901095    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11d41482-3638-4d28-bef1-e2abd6027e44" path="/var/lib/kubelet/pods/11d41482-3638-4d28-bef1-e2abd6027e44/volumes"
Sep 09 12:25:26.011482 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:26.011445    2585 generic.go:356] "Generic (PLEG): container finished" podID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerID="06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1" exitCode=0
Sep 09 12:25:26.011657 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:26.011517    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" event={"ID":"d480be69-2c10-4eeb-9819-56ac690fb8c4","Type":"ContainerDied","Data":"06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1"}
Sep 09 12:25:26.012017 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:26.012003    2585 scope.go:117] "RemoveContainer" containerID="06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1"
Sep 09 12:25:26.013166 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:26.013140    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:25:26.013276 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:26.013219    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:25:26.013340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:26.013288    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:25:27.019087 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:27.019056    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" event={"ID":"d480be69-2c10-4eeb-9819-56ac690fb8c4","Type":"ContainerStarted","Data":"930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5"}
Sep 09 12:25:27.708266 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:27.708230    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:25:27.709315 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:27.709291    2585 patch_prober.go:28] interesting pod/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 09 12:25:27.709381 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:27.709345    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 09 12:25:28.708355 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:28.708302    2585 patch_prober.go:28] interesting pod/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 09 12:25:28.709058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:28.708392    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 09 12:25:29.708553 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.708519    2585 patch_prober.go:28] interesting pod/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 09 12:25:29.708912 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.708571    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 09 12:25:29.798245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.798178    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:25:29.848486 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.848429    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:25:29.925122 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925094    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-data\") pod \"2b126680-71c8-4809-af0c-f2b0a0840b92\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") "
Sep 09 12:25:29.925282 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925137    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-socket\") pod \"2b126680-71c8-4809-af0c-f2b0a0840b92\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") "
Sep 09 12:25:29.925282 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925178    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-token\") pod \"2b126680-71c8-4809-af0c-f2b0a0840b92\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") "
Sep 09 12:25:29.925282 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925262    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-podinfo\") pod \"2b126680-71c8-4809-af0c-f2b0a0840b92\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") "
Sep 09 12:25:29.925446 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925289    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-certs\") pod \"2b126680-71c8-4809-af0c-f2b0a0840b92\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") "
Sep 09 12:25:29.925446 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925396    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-data" (OuterVolumeSpecName: "istio-data") pod "2b126680-71c8-4809-af0c-f2b0a0840b92" (UID: "2b126680-71c8-4809-af0c-f2b0a0840b92"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:29.925446 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925414    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2b126680-71c8-4809-af0c-f2b0a0840b92" (UID: "2b126680-71c8-4809-af0c-f2b0a0840b92"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:29.925446 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925421    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-envoy\") pod \"2b126680-71c8-4809-af0c-f2b0a0840b92\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") "
Sep 09 12:25:29.925660 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925469    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2h75\" (UniqueName: \"kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-kube-api-access-x2h75\") pod \"2b126680-71c8-4809-af0c-f2b0a0840b92\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") "
Sep 09 12:25:29.925660 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925503    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b126680-71c8-4809-af0c-f2b0a0840b92-istiod-ca-cert\") pod \"2b126680-71c8-4809-af0c-f2b0a0840b92\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") "
Sep 09 12:25:29.925660 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925549    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-credential-socket\") pod \"2b126680-71c8-4809-af0c-f2b0a0840b92\" (UID: \"2b126680-71c8-4809-af0c-f2b0a0840b92\") "
Sep 09 12:25:29.925660 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925553    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2b126680-71c8-4809-af0c-f2b0a0840b92" (UID: "2b126680-71c8-4809-af0c-f2b0a0840b92"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:29.925903 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925883    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2b126680-71c8-4809-af0c-f2b0a0840b92" (UID: "2b126680-71c8-4809-af0c-f2b0a0840b92"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:29.925903 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.925884    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b126680-71c8-4809-af0c-f2b0a0840b92-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2b126680-71c8-4809-af0c-f2b0a0840b92" (UID: "2b126680-71c8-4809-af0c-f2b0a0840b92"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:25:29.926184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.926166    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:29.926275 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.926196    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b126680-71c8-4809-af0c-f2b0a0840b92-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:29.926275 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.926238    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:29.926275 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.926250    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:29.926275 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.926263    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:29.927871 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.927845    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2b126680-71c8-4809-af0c-f2b0a0840b92" (UID: "2b126680-71c8-4809-af0c-f2b0a0840b92"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:29.927871 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.927860    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-token" (OuterVolumeSpecName: "istio-token") pod "2b126680-71c8-4809-af0c-f2b0a0840b92" (UID: "2b126680-71c8-4809-af0c-f2b0a0840b92"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:25:29.928022 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.927977    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2b126680-71c8-4809-af0c-f2b0a0840b92" (UID: "2b126680-71c8-4809-af0c-f2b0a0840b92"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:25:29.928022 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:29.927981    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-kube-api-access-x2h75" (OuterVolumeSpecName: "kube-api-access-x2h75") pod "2b126680-71c8-4809-af0c-f2b0a0840b92" (UID: "2b126680-71c8-4809-af0c-f2b0a0840b92"). InnerVolumeSpecName "kube-api-access-x2h75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:25:30.027662 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.027629    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:30.027662 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.027655    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:30.027662 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.027665    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b126680-71c8-4809-af0c-f2b0a0840b92-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:30.027904 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.027674    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2h75\" (UniqueName: \"kubernetes.io/projected/2b126680-71c8-4809-af0c-f2b0a0840b92-kube-api-access-x2h75\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:30.037978 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.037949    2585 generic.go:356] "Generic (PLEG): container finished" podID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerID="ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600" exitCode=0
Sep 09 12:25:30.038123 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.037983    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" event={"ID":"2b126680-71c8-4809-af0c-f2b0a0840b92","Type":"ContainerDied","Data":"ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600"}
Sep 09 12:25:30.038123 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.038011    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"
Sep 09 12:25:30.038123 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.038022    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95" event={"ID":"2b126680-71c8-4809-af0c-f2b0a0840b92","Type":"ContainerDied","Data":"fc05b558bc38de163ab62a2e88677fe9118bb85b1b02f793d8f37ba1e4e39135"}
Sep 09 12:25:30.038123 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.038042    2585 scope.go:117] "RemoveContainer" containerID="ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600"
Sep 09 12:25:30.047642 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.047624    2585 scope.go:117] "RemoveContainer" containerID="78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73"
Sep 09 12:25:30.075898 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.073850    2585 scope.go:117] "RemoveContainer" containerID="ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600"
Sep 09 12:25:30.075898 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:25:30.075078    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600\": container with ID starting with ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600 not found: ID does not exist" containerID="ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600"
Sep 09 12:25:30.075898 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.075112    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600"} err="failed to get container status \"ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600\": rpc error: code = NotFound desc = could not find container \"ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600\": container with ID starting with ef42ae345c6910869fdbd07851e197f469a6a45e5fd7cf142d7b901d925ab600 not found: ID does not exist"
Sep 09 12:25:30.075898 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.075135    2585 scope.go:117] "RemoveContainer" containerID="78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73"
Sep 09 12:25:30.077937 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:25:30.077900    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73\": container with ID starting with 78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73 not found: ID does not exist" containerID="78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73"
Sep 09 12:25:30.078059 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.077948    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73"} err="failed to get container status \"78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73\": rpc error: code = NotFound desc = could not find container \"78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73\": container with ID starting with 78089664e27fffbb90d0c033b3a7c3377a565785de7e1cf99e450623a0ebfe73 not found: ID does not exist"
Sep 09 12:25:30.079228 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.079206    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"]
Sep 09 12:25:30.084465 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.084444    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c801b865-openshift-default-6d58b8df8-x2c95"]
Sep 09 12:25:30.707968 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.707929    2585 patch_prober.go:28] interesting pod/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 09 12:25:30.708214 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:30.707984    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 09 12:25:31.708682 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:31.708637    2585 patch_prober.go:28] interesting pod/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Sep 09 12:25:31.709105 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:31.708721    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Sep 09 12:25:31.900222 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:31.900192    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" path="/var/lib/kubelet/pods/2b126680-71c8-4809-af0c-f2b0a0840b92/volumes"
Sep 09 12:25:32.713440 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:32.713401    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:25:32.714493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:32.714474    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:25:53.387320 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.387284    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"]
Sep 09 12:25:53.387760 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.387746    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d41482-3638-4d28-bef1-e2abd6027e44" containerName="revoke-keys"
Sep 09 12:25:53.387760 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.387761    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d41482-3638-4d28-bef1-e2abd6027e44" containerName="revoke-keys"
Sep 09 12:25:53.387866 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.387774    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy"
Sep 09 12:25:53.387866 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.387780    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy"
Sep 09 12:25:53.387866 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.387792    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy"
Sep 09 12:25:53.387866 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.387798    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy"
Sep 09 12:25:53.387866 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.387864    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy"
Sep 09 12:25:53.388018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.387879    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b126680-71c8-4809-af0c-f2b0a0840b92" containerName="istio-proxy"
Sep 09 12:25:53.388018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.387887    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d41482-3638-4d28-bef1-e2abd6027e44" containerName="revoke-keys"
Sep 09 12:25:53.391097 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.391080    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"
Sep 09 12:25:53.393502 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.393476    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:25:53.393605 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.393477    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:25:53.398487 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.398459    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"]
Sep 09 12:25:53.461869 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.461831    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6\" (UID: \"2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"
Sep 09 12:25:53.563290 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.563248    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6\" (UID: \"2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"
Sep 09 12:25:53.563991 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.563970    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6\" (UID: \"2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"
Sep 09 12:25:53.702928 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.702846    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"
Sep 09 12:25:53.981406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:53.981327    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"]
Sep 09 12:25:53.984411 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:25:53.984388    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f9b9bf9_aa4f_4ca5_ad28_ae5e0d2434fb.slice/crio-8541574ffa4ccc1ea4097bc6b05d34b6962c41acbff8776b38e2c6c107206314 WatchSource:0}: Error finding container 8541574ffa4ccc1ea4097bc6b05d34b6962c41acbff8776b38e2c6c107206314: Status 404 returned error can't find the container with id 8541574ffa4ccc1ea4097bc6b05d34b6962c41acbff8776b38e2c6c107206314
Sep 09 12:25:54.176477 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:54.176442    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6" event={"ID":"2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb","Type":"ContainerStarted","Data":"8541574ffa4ccc1ea4097bc6b05d34b6962c41acbff8776b38e2c6c107206314"}
Sep 09 12:25:55.182929 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:55.182896    2585 generic.go:356] "Generic (PLEG): container finished" podID="2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb" containerID="86d1381a8d23ee347859c689b0121d0f287059f0d11d67950520ac4c0f19de9e" exitCode=0
Sep 09 12:25:55.183313 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:55.182980    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6" event={"ID":"2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb","Type":"ContainerDied","Data":"86d1381a8d23ee347859c689b0121d0f287059f0d11d67950520ac4c0f19de9e"}
Sep 09 12:25:56.320968 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:56.320942    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"
Sep 09 12:25:56.390591 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:56.390556    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb-maas-api-service-ca\") pod \"2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb\" (UID: \"2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb\") "
Sep 09 12:25:56.390915 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:56.390890    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb" (UID: "2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:25:56.492106 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:56.492023    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:25:57.194133 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:57.194105    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"
Sep 09 12:25:57.194133 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:57.194114    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6" event={"ID":"2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb","Type":"ContainerDied","Data":"8541574ffa4ccc1ea4097bc6b05d34b6962c41acbff8776b38e2c6c107206314"}
Sep 09 12:25:57.194341 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:57.194157    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8541574ffa4ccc1ea4097bc6b05d34b6962c41acbff8776b38e2c6c107206314"
Sep 09 12:25:59.363148 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:59.363109    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"]
Sep 09 12:25:59.366132 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:59.366104    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-669203b2-gk2w6"]
Sep 09 12:25:59.899971 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:25:59.899939    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb" path="/var/lib/kubelet/pods/2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb/volumes"
Sep 09 12:26:00.347417 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.347380    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"]
Sep 09 12:26:00.347847 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.347833    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb" containerName="revoke-keys"
Sep 09 12:26:00.347847 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.347849    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb" containerName="revoke-keys"
Sep 09 12:26:00.347944 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.347935    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f9b9bf9-aa4f-4ca5-ad28-ae5e0d2434fb" containerName="revoke-keys"
Sep 09 12:26:00.350868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.350848    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.353315 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.353297    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8ee5335d-openshift-default-dockercfg-wgfkn\""
Sep 09 12:26:00.364001 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.363978    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"]
Sep 09 12:26:00.429289 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.429238    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-credential-socket\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.429547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.429312    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istiod-ca-cert\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.429547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.429377    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-data\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.429547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.429414    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-token\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.429547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.429487    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-envoy\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.429547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.429525    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-certs\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.429861 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.429549    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxn7p\" (UniqueName: \"kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-kube-api-access-bxn7p\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.429861 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.429575    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-socket\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.429861 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.429617    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-podinfo\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.530739 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.530689    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istiod-ca-cert\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.530905 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.530771    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-data\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.530905 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.530807    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-token\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.530905 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.530874    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-envoy\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.531064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.530910    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-certs\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.531064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.530932    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxn7p\" (UniqueName: \"kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-kube-api-access-bxn7p\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.531064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.530956    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-socket\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.531064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.530997    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-podinfo\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.531064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.531052    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-credential-socket\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.531327 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.531239    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-data\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.531327 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.531301    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-certs\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.531412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.531388    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-socket\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.531474 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.531442    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-credential-socket\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.531531 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.531510    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istiod-ca-cert\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.533612 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.533590    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-envoy\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.533783 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.533766    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-podinfo\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.539383 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.539358    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-token\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.539520 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.539502    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxn7p\" (UniqueName: \"kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-kube-api-access-bxn7p\") pod \"e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") " pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.662846 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.662757    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:00.946265 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.946189    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"]
Sep 09 12:26:00.952893 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:26:00.952849    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee6020c6_31e7_4b41_a531_3eabfdc2125f.slice/crio-c97412df39143ae3efce9ec08ac374dddc89a88a1722cac723db9e14718ae38d WatchSource:0}: Error finding container c97412df39143ae3efce9ec08ac374dddc89a88a1722cac723db9e14718ae38d: Status 404 returned error can't find the container with id c97412df39143ae3efce9ec08ac374dddc89a88a1722cac723db9e14718ae38d
Sep 09 12:26:00.955946 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.955910    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:26:00.956045 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.955982    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:26:00.956045 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:00.956018    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:26:01.220868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:01.220779    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" event={"ID":"ee6020c6-31e7-4b41-a531-3eabfdc2125f","Type":"ContainerStarted","Data":"bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534"}
Sep 09 12:26:01.220868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:01.220839    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" event={"ID":"ee6020c6-31e7-4b41-a531-3eabfdc2125f","Type":"ContainerStarted","Data":"c97412df39143ae3efce9ec08ac374dddc89a88a1722cac723db9e14718ae38d"}
Sep 09 12:26:01.242016 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:01.241956    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" podStartSLOduration=1.241935491 podStartE2EDuration="1.241935491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:01.240512662 +0000 UTC m=+905.882131859" watchObservedRunningTime="2026-09-09 12:26:01.241935491 +0000 UTC m=+905.883554698"
Sep 09 12:26:01.663287 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:01.663244    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:02.668517 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:02.668485    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:03.232921 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:03.232887    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:03.233869 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:03.233852    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:11.199770 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.199732    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"]
Sep 09 12:26:11.203852 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.203832    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.207130 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.207102    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-bc0270\"/\"openshift-service-ca.crt\""
Sep 09 12:26:11.207251 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.207140    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-bc0270\"/\"simulator-w4-bc0270-kserve-self-signed-certs\""
Sep 09 12:26:11.207251 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.207169    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-bc0270\"/\"kube-root-ca.crt\""
Sep 09 12:26:11.207495 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.207337    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-bc0270\"/\"default-dockercfg-k469g\""
Sep 09 12:26:11.214534 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.214507    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"]
Sep 09 12:26:11.337610 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.337573    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-model-cache\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.337822 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.337634    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf8hb\" (UniqueName: \"kubernetes.io/projected/628822ce-5179-434a-9479-a307bb2c91c9-kube-api-access-mf8hb\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.337822 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.337741    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-tmp-dir\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.337822 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.337778    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-dshm\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.337964 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.337826    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-home\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.337964 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.337860    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628822ce-5179-434a-9479-a307bb2c91c9-tls-certs\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.439117 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.439077    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-model-cache\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.439321 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.439153    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf8hb\" (UniqueName: \"kubernetes.io/projected/628822ce-5179-434a-9479-a307bb2c91c9-kube-api-access-mf8hb\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.439321 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.439195    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-tmp-dir\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.439321 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.439223    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-dshm\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.439321 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.439278    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-home\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.439321 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.439318    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628822ce-5179-434a-9479-a307bb2c91c9-tls-certs\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.439620 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.439472    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-model-cache\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.439620 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.439535    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-tmp-dir\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.439620 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.439583    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-home\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.441591 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.441561    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-dshm\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.441976 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.441955    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628822ce-5179-434a-9479-a307bb2c91c9-tls-certs\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.448470 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.448448    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf8hb\" (UniqueName: \"kubernetes.io/projected/628822ce-5179-434a-9479-a307bb2c91c9-kube-api-access-mf8hb\") pod \"simulator-w4-bc0270-kserve-7b66b78459-hgbwj\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") " pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.514511 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.514473    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:11.791494 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:11.791465    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"]
Sep 09 12:26:11.793492 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:26:11.793465    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod628822ce_5179_434a_9479_a307bb2c91c9.slice/crio-e7c1f4e05df88aa6ddb7f8a61d6c53dc75ebbc24664b7edee380f535f76eae23 WatchSource:0}: Error finding container e7c1f4e05df88aa6ddb7f8a61d6c53dc75ebbc24664b7edee380f535f76eae23: Status 404 returned error can't find the container with id e7c1f4e05df88aa6ddb7f8a61d6c53dc75ebbc24664b7edee380f535f76eae23
Sep 09 12:26:12.283896 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:12.283851    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj" event={"ID":"628822ce-5179-434a-9479-a307bb2c91c9","Type":"ContainerStarted","Data":"6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17"}
Sep 09 12:26:12.283896 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:12.283895    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj" event={"ID":"628822ce-5179-434a-9479-a307bb2c91c9","Type":"ContainerStarted","Data":"e7c1f4e05df88aa6ddb7f8a61d6c53dc75ebbc24664b7edee380f535f76eae23"}
Sep 09 12:26:12.301002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:12.300955    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj" podStartSLOduration=1.300940925 podStartE2EDuration="1.300940925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:12.299101774 +0000 UTC m=+916.940720977" watchObservedRunningTime="2026-09-09 12:26:12.300940925 +0000 UTC m=+916.942560129"
Sep 09 12:26:21.515321 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:21.515282    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:21.515842 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:21.515818    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:21.527996 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:21.527975    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:22.354438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:22.354155    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:26:26.366291 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:26.366258    2585 generic.go:356] "Generic (PLEG): container finished" podID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerID="bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534" exitCode=0
Sep 09 12:26:26.366723 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:26.366328    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" event={"ID":"ee6020c6-31e7-4b41-a531-3eabfdc2125f","Type":"ContainerDied","Data":"bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534"}
Sep 09 12:26:26.366811 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:26.366796    2585 scope.go:117] "RemoveContainer" containerID="bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534"
Sep 09 12:26:26.367976 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:26.367947    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:26:26.368035 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:26.368015    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:26:26.368070 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:26.368046    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:26:27.374064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:27.374034    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" event={"ID":"ee6020c6-31e7-4b41-a531-3eabfdc2125f","Type":"ContainerStarted","Data":"39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50"}
Sep 09 12:26:27.663422 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:27.663338    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:27.663572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:27.663525    2585 patch_prober.go:28] interesting pod/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 12:26:27.663613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:27.663578    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 12:26:28.532146 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.532113    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"]
Sep 09 12:26:28.535866 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.535849    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"
Sep 09 12:26:28.538067 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.538040    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:26:28.538178 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.538107    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:26:28.543035 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.543007    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"]
Sep 09 12:26:28.630994 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.630957    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f173ef-216a-46a7-939c-576f90c50e92-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk\" (UID: \"86f173ef-216a-46a7-939c-576f90c50e92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"
Sep 09 12:26:28.663741 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.663678    2585 patch_prober.go:28] interesting pod/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 12:26:28.664024 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.663999    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 12:26:28.731606 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.731568    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f173ef-216a-46a7-939c-576f90c50e92-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk\" (UID: \"86f173ef-216a-46a7-939c-576f90c50e92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"
Sep 09 12:26:28.732191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.732173    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f173ef-216a-46a7-939c-576f90c50e92-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk\" (UID: \"86f173ef-216a-46a7-939c-576f90c50e92\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"
Sep 09 12:26:28.847331 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:28.847238    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"
Sep 09 12:26:29.140793 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:29.140765    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"]
Sep 09 12:26:29.142986 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:26:29.142960    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f173ef_216a_46a7_939c_576f90c50e92.slice/crio-450b3c08dc7821646f410817842b6afb3c53374ce4d830db099bfe49858b1a56 WatchSource:0}: Error finding container 450b3c08dc7821646f410817842b6afb3c53374ce4d830db099bfe49858b1a56: Status 404 returned error can't find the container with id 450b3c08dc7821646f410817842b6afb3c53374ce4d830db099bfe49858b1a56
Sep 09 12:26:29.390689 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:29.390605    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk" event={"ID":"86f173ef-216a-46a7-939c-576f90c50e92","Type":"ContainerStarted","Data":"450b3c08dc7821646f410817842b6afb3c53374ce4d830db099bfe49858b1a56"}
Sep 09 12:26:29.663983 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:29.663476    2585 patch_prober.go:28] interesting pod/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 12:26:29.663983 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:29.663542    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 12:26:30.397983 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:30.397947    2585 generic.go:356] "Generic (PLEG): container finished" podID="86f173ef-216a-46a7-939c-576f90c50e92" containerID="ca121154d2f7f245b0a0b6375f731ec69a6593e9456abc591674f616ddf0d9af" exitCode=0
Sep 09 12:26:30.398148 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:30.398038    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk" event={"ID":"86f173ef-216a-46a7-939c-576f90c50e92","Type":"ContainerDied","Data":"ca121154d2f7f245b0a0b6375f731ec69a6593e9456abc591674f616ddf0d9af"}
Sep 09 12:26:30.663557 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:30.663522    2585 patch_prober.go:28] interesting pod/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 12:26:30.663876 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:30.663842    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 12:26:31.281946 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.281912    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"]
Sep 09 12:26:31.286892 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.286870    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.290672 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.290474    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e49066\"/\"openshift-service-ca.crt\""
Sep 09 12:26:31.290672 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.290511    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e49066\"/\"test-model-e49066-kserve-self-signed-certs\""
Sep 09 12:26:31.290672 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.290553    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e49066\"/\"default-dockercfg-8jmzt\""
Sep 09 12:26:31.291046 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.291023    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e49066\"/\"kube-root-ca.crt\""
Sep 09 12:26:31.296441 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.296420    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"]
Sep 09 12:26:31.361297 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.361255    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-model-cache\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.361631 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.361608    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tmp-dir\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.361772 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.361756    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-dshm\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.361954 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.361932    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngzzz\" (UniqueName: \"kubernetes.io/projected/364eb190-c9e4-4f34-8b8f-efd15708e5f6-kube-api-access-ngzzz\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.362106 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.362090    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tls-certs\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.362294 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.362280    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-home\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.498920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.466902    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngzzz\" (UniqueName: \"kubernetes.io/projected/364eb190-c9e4-4f34-8b8f-efd15708e5f6-kube-api-access-ngzzz\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.498920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.466969    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tls-certs\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.498920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.467055    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-home\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.498920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.467089    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-model-cache\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.498920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.467146    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tmp-dir\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.498920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.467169    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-dshm\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.498920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.467948    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-home\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.498920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.476014    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-model-cache\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.498920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.476869    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tmp-dir\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.498920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.488547    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngzzz\" (UniqueName: \"kubernetes.io/projected/364eb190-c9e4-4f34-8b8f-efd15708e5f6-kube-api-access-ngzzz\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.499635 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.499573    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-dshm\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.504769 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.502824    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tls-certs\") pod \"test-model-e49066-kserve-6b7c9c7dd9-fjrnx\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") " pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.586574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.586548    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"
Sep 09 12:26:31.597740 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.597694    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:31.678733 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.676722    2585 patch_prober.go:28] interesting pod/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 12:26:31.678733 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.676780    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 12:26:31.777159 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.777126    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f173ef-216a-46a7-939c-576f90c50e92-maas-api-service-ca\") pod \"86f173ef-216a-46a7-939c-576f90c50e92\" (UID: \"86f173ef-216a-46a7-939c-576f90c50e92\") "
Sep 09 12:26:31.777497 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.777469    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86f173ef-216a-46a7-939c-576f90c50e92-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "86f173ef-216a-46a7-939c-576f90c50e92" (UID: "86f173ef-216a-46a7-939c-576f90c50e92"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:26:31.878077 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.877996    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f173ef-216a-46a7-939c-576f90c50e92-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:26:31.892097 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:31.892071    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"]
Sep 09 12:26:31.893851 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:26:31.893825    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod364eb190_c9e4_4f34_8b8f_efd15708e5f6.slice/crio-52fa441c0b08908e3f2df70def6aa8a9241e7b20610ff8d10e6220aad5b55edf WatchSource:0}: Error finding container 52fa441c0b08908e3f2df70def6aa8a9241e7b20610ff8d10e6220aad5b55edf: Status 404 returned error can't find the container with id 52fa441c0b08908e3f2df70def6aa8a9241e7b20610ff8d10e6220aad5b55edf
Sep 09 12:26:32.411358 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:32.411326    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"
Sep 09 12:26:32.411863 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:32.411363    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk" event={"ID":"86f173ef-216a-46a7-939c-576f90c50e92","Type":"ContainerDied","Data":"450b3c08dc7821646f410817842b6afb3c53374ce4d830db099bfe49858b1a56"}
Sep 09 12:26:32.411863 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:32.411437    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="450b3c08dc7821646f410817842b6afb3c53374ce4d830db099bfe49858b1a56"
Sep 09 12:26:32.413163 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:32.413128    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx" event={"ID":"364eb190-c9e4-4f34-8b8f-efd15708e5f6","Type":"ContainerStarted","Data":"ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d"}
Sep 09 12:26:32.413286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:32.413171    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx" event={"ID":"364eb190-c9e4-4f34-8b8f-efd15708e5f6","Type":"ContainerStarted","Data":"52fa441c0b08908e3f2df70def6aa8a9241e7b20610ff8d10e6220aad5b55edf"}
Sep 09 12:26:32.433123 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:32.433081    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx" podStartSLOduration=1.433068092 podStartE2EDuration="1.433068092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:32.432326312 +0000 UTC m=+937.073945501" watchObservedRunningTime="2026-09-09 12:26:32.433068092 +0000 UTC m=+937.074687295"
Sep 09 12:26:32.664220 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:32.664128    2585 patch_prober.go:28] interesting pod/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 12:26:32.664220 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:32.664198    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 12:26:33.233161 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:33.233122    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:33.667949 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:33.667919    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:33.668880 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:33.668859    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:34.559641 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:34.559602    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"]
Sep 09 12:26:34.561683 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:34.561663    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ee5335d-zw8sk"]
Sep 09 12:26:35.798548 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:35.798511    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"]
Sep 09 12:26:35.799025 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:35.798752    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy" containerID="cri-o://39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50" gracePeriod=30
Sep 09 12:26:35.903857 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:35.903819    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86f173ef-216a-46a7-939c-576f90c50e92" path="/var/lib/kubelet/pods/86f173ef-216a-46a7-939c-576f90c50e92/volumes"
Sep 09 12:26:41.066976 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.066907    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:41.174488 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174447    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istiod-ca-cert\") pod \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") "
Sep 09 12:26:41.174691 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174504    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-credential-socket\") pod \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") "
Sep 09 12:26:41.174691 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174540    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-certs\") pod \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") "
Sep 09 12:26:41.174691 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174567    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-envoy\") pod \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") "
Sep 09 12:26:41.174691 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174614    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-socket\") pod \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") "
Sep 09 12:26:41.174691 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174676    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-data\") pod \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") "
Sep 09 12:26:41.174998 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174777    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxn7p\" (UniqueName: \"kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-kube-api-access-bxn7p\") pod \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") "
Sep 09 12:26:41.174998 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174815    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-token\") pod \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") "
Sep 09 12:26:41.174998 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174853    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-podinfo\") pod \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\" (UID: \"ee6020c6-31e7-4b41-a531-3eabfdc2125f\") "
Sep 09 12:26:41.174998 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174866    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee6020c6-31e7-4b41-a531-3eabfdc2125f" (UID: "ee6020c6-31e7-4b41-a531-3eabfdc2125f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:26:41.174998 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.174921    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee6020c6-31e7-4b41-a531-3eabfdc2125f" (UID: "ee6020c6-31e7-4b41-a531-3eabfdc2125f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:26:41.175245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.175017    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee6020c6-31e7-4b41-a531-3eabfdc2125f" (UID: "ee6020c6-31e7-4b41-a531-3eabfdc2125f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:26:41.175245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.175153    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee6020c6-31e7-4b41-a531-3eabfdc2125f" (UID: "ee6020c6-31e7-4b41-a531-3eabfdc2125f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:26:41.175245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.175168    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee6020c6-31e7-4b41-a531-3eabfdc2125f" (UID: "ee6020c6-31e7-4b41-a531-3eabfdc2125f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:26:41.175387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.175290    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:26:41.175387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.175310    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:26:41.175387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.175324    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:26:41.175387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.175337    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:26:41.175387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.175350    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:26:41.177251 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.177228    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee6020c6-31e7-4b41-a531-3eabfdc2125f" (UID: "ee6020c6-31e7-4b41-a531-3eabfdc2125f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:26:41.177344 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.177279    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee6020c6-31e7-4b41-a531-3eabfdc2125f" (UID: "ee6020c6-31e7-4b41-a531-3eabfdc2125f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:26:41.177669 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.177647    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee6020c6-31e7-4b41-a531-3eabfdc2125f" (UID: "ee6020c6-31e7-4b41-a531-3eabfdc2125f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:26:41.177669 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.177646    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-kube-api-access-bxn7p" (OuterVolumeSpecName: "kube-api-access-bxn7p") pod "ee6020c6-31e7-4b41-a531-3eabfdc2125f" (UID: "ee6020c6-31e7-4b41-a531-3eabfdc2125f"). InnerVolumeSpecName "kube-api-access-bxn7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:26:41.276259 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.276227    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxn7p\" (UniqueName: \"kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-kube-api-access-bxn7p\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:26:41.276259 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.276257    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:26:41.276259 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.276267    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:26:41.276259 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.276275    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee6020c6-31e7-4b41-a531-3eabfdc2125f-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:26:41.477657 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.477570    2585 generic.go:356] "Generic (PLEG): container finished" podID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerID="39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50" exitCode=0
Sep 09 12:26:41.477657 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.477618    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" event={"ID":"ee6020c6-31e7-4b41-a531-3eabfdc2125f","Type":"ContainerDied","Data":"39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50"}
Sep 09 12:26:41.477657 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.477630    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"
Sep 09 12:26:41.477913 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.477670    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7" event={"ID":"ee6020c6-31e7-4b41-a531-3eabfdc2125f","Type":"ContainerDied","Data":"c97412df39143ae3efce9ec08ac374dddc89a88a1722cac723db9e14718ae38d"}
Sep 09 12:26:41.477913 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.477724    2585 scope.go:117] "RemoveContainer" containerID="39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50"
Sep 09 12:26:41.487758 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.487741    2585 scope.go:117] "RemoveContainer" containerID="bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534"
Sep 09 12:26:41.498871 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.498852    2585 scope.go:117] "RemoveContainer" containerID="39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50"
Sep 09 12:26:41.499190 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:26:41.499168    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50\": container with ID starting with 39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50 not found: ID does not exist" containerID="39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50"
Sep 09 12:26:41.499243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.499199    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50"} err="failed to get container status \"39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50\": rpc error: code = NotFound desc = could not find container \"39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50\": container with ID starting with 39940f70e89d54b12427bfdc591e046b130292fefbcad24a76e1175d6426cb50 not found: ID does not exist"
Sep 09 12:26:41.499243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.499218    2585 scope.go:117] "RemoveContainer" containerID="bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534"
Sep 09 12:26:41.499495 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:26:41.499477    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534\": container with ID starting with bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534 not found: ID does not exist" containerID="bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534"
Sep 09 12:26:41.499554 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.499506    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534"} err="failed to get container status \"bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534\": rpc error: code = NotFound desc = could not find container \"bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534\": container with ID starting with bc294abb57eba53d47e37ad3c21438229e651ee5fcc88c42f95d18898209b534 not found: ID does not exist"
Sep 09 12:26:41.500885 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.500866    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"]
Sep 09 12:26:41.504422 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.504401    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8ee5335d-openshift-default-6fcfb5b784-gxdm7"]
Sep 09 12:26:41.598483 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.598452    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:41.598642 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.598524    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:41.611409 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.611388    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:41.901066 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:41.901031    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" path="/var/lib/kubelet/pods/ee6020c6-31e7-4b41-a531-3eabfdc2125f/volumes"
Sep 09 12:26:42.495627 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:42.495598    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:26:56.611149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.611109    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"]
Sep 09 12:26:56.611786 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.611764    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy"
Sep 09 12:26:56.611786 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.611787    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy"
Sep 09 12:26:56.611944 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.611815    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86f173ef-216a-46a7-939c-576f90c50e92" containerName="revoke-keys"
Sep 09 12:26:56.611944 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.611824    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="86f173ef-216a-46a7-939c-576f90c50e92" containerName="revoke-keys"
Sep 09 12:26:56.612047 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.611949    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="86f173ef-216a-46a7-939c-576f90c50e92" containerName="revoke-keys"
Sep 09 12:26:56.612047 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.611965    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy"
Sep 09 12:26:56.616398 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.616375    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"
Sep 09 12:26:56.619026 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.619000    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:26:56.619887 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.619842    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:26:56.621076 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.621054    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"]
Sep 09 12:26:56.732191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.732158    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfc68881-3a91-4956-8da1-09e922e9f574-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-a1e1f1-tg454\" (UID: \"bfc68881-3a91-4956-8da1-09e922e9f574\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"
Sep 09 12:26:56.833839 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.833803    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfc68881-3a91-4956-8da1-09e922e9f574-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-a1e1f1-tg454\" (UID: \"bfc68881-3a91-4956-8da1-09e922e9f574\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"
Sep 09 12:26:56.834406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.834384    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfc68881-3a91-4956-8da1-09e922e9f574-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-a1e1f1-tg454\" (UID: \"bfc68881-3a91-4956-8da1-09e922e9f574\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"
Sep 09 12:26:56.929625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:56.929543    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"
Sep 09 12:26:57.201175 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:57.201148    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"]
Sep 09 12:26:57.202726 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:26:57.202685    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc68881_3a91_4956_8da1_09e922e9f574.slice/crio-a230f56304a7568946fcd0e744cafdad9e77d69579a8a464447cdc7217bf5322 WatchSource:0}: Error finding container a230f56304a7568946fcd0e744cafdad9e77d69579a8a464447cdc7217bf5322: Status 404 returned error can't find the container with id a230f56304a7568946fcd0e744cafdad9e77d69579a8a464447cdc7217bf5322
Sep 09 12:26:57.574363 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:57.574325    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454" event={"ID":"bfc68881-3a91-4956-8da1-09e922e9f574","Type":"ContainerStarted","Data":"f6994e4f4190b2315437ac0ebadd9671e1379d5dae771093873444f6e6160812"}
Sep 09 12:26:57.574550 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:57.574423    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454" event={"ID":"bfc68881-3a91-4956-8da1-09e922e9f574","Type":"ContainerStarted","Data":"a230f56304a7568946fcd0e744cafdad9e77d69579a8a464447cdc7217bf5322"}
Sep 09 12:26:57.588921 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:57.588855    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454" podStartSLOduration=1.5888355619999999 podStartE2EDuration="1.588835562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:57.587743176 +0000 UTC m=+962.229362381" watchObservedRunningTime="2026-09-09 12:26:57.588835562 +0000 UTC m=+962.230454766"
Sep 09 12:26:58.581140 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:58.581101    2585 generic.go:356] "Generic (PLEG): container finished" podID="bfc68881-3a91-4956-8da1-09e922e9f574" containerID="f6994e4f4190b2315437ac0ebadd9671e1379d5dae771093873444f6e6160812" exitCode=0
Sep 09 12:26:58.581529 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:58.581151    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454" event={"ID":"bfc68881-3a91-4956-8da1-09e922e9f574","Type":"ContainerDied","Data":"f6994e4f4190b2315437ac0ebadd9671e1379d5dae771093873444f6e6160812"}
Sep 09 12:26:59.728265 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:59.728234    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"
Sep 09 12:26:59.868265 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:59.868179    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfc68881-3a91-4956-8da1-09e922e9f574-maas-api-service-ca\") pod \"bfc68881-3a91-4956-8da1-09e922e9f574\" (UID: \"bfc68881-3a91-4956-8da1-09e922e9f574\") "
Sep 09 12:26:59.868626 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:59.868599    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfc68881-3a91-4956-8da1-09e922e9f574-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bfc68881-3a91-4956-8da1-09e922e9f574" (UID: "bfc68881-3a91-4956-8da1-09e922e9f574"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:26:59.969448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:26:59.969410    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfc68881-3a91-4956-8da1-09e922e9f574-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:00.592956 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:00.592928    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"
Sep 09 12:27:00.593153 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:00.592928    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454" event={"ID":"bfc68881-3a91-4956-8da1-09e922e9f574","Type":"ContainerDied","Data":"a230f56304a7568946fcd0e744cafdad9e77d69579a8a464447cdc7217bf5322"}
Sep 09 12:27:00.593153 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:00.593048    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a230f56304a7568946fcd0e744cafdad9e77d69579a8a464447cdc7217bf5322"
Sep 09 12:27:03.576668 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.576628    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"]
Sep 09 12:27:03.577167 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.577078    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfc68881-3a91-4956-8da1-09e922e9f574" containerName="revoke-keys"
Sep 09 12:27:03.577167 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.577090    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfc68881-3a91-4956-8da1-09e922e9f574" containerName="revoke-keys"
Sep 09 12:27:03.577167 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.577109    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy"
Sep 09 12:27:03.577167 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.577115    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy"
Sep 09 12:27:03.577320 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.577189    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee6020c6-31e7-4b41-a531-3eabfdc2125f" containerName="istio-proxy"
Sep 09 12:27:03.577320 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.577201    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfc68881-3a91-4956-8da1-09e922e9f574" containerName="revoke-keys"
Sep 09 12:27:03.580387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.580369    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"
Sep 09 12:27:03.582519 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.582499    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:27:03.582662 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.582550    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:27:03.588759 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.588285    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"]
Sep 09 12:27:03.708776 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.708725    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml\" (UID: \"5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"
Sep 09 12:27:03.810210 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.810175    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml\" (UID: \"5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"
Sep 09 12:27:03.810860 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.810840    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml\" (UID: \"5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"
Sep 09 12:27:03.893269 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:03.893169    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"
Sep 09 12:27:04.169818 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:04.169793    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"]
Sep 09 12:27:04.172105 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:27:04.172077    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a03c6fd_a1a2_450c_af5c_1f0e0b0e3aa0.slice/crio-10781367cdab34637daf644160fe566d7dfa9f7a5745c64679803cd6a5d2941a WatchSource:0}: Error finding container 10781367cdab34637daf644160fe566d7dfa9f7a5745c64679803cd6a5d2941a: Status 404 returned error can't find the container with id 10781367cdab34637daf644160fe566d7dfa9f7a5745c64679803cd6a5d2941a
Sep 09 12:27:04.623236 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:04.623199    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml" event={"ID":"5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0","Type":"ContainerStarted","Data":"4b53275ec37b9d23142c4c2503ebe6ea2a5b3681d3fce3fa26d203da57b98467"}
Sep 09 12:27:04.623688 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:04.623257    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml" event={"ID":"5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0","Type":"ContainerStarted","Data":"10781367cdab34637daf644160fe566d7dfa9f7a5745c64679803cd6a5d2941a"}
Sep 09 12:27:04.640283 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:04.640236    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml" podStartSLOduration=1.6402211389999999 podStartE2EDuration="1.640221139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:27:04.638831855 +0000 UTC m=+969.280451059" watchObservedRunningTime="2026-09-09 12:27:04.640221139 +0000 UTC m=+969.281840396"
Sep 09 12:27:05.629570 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:05.629537    2585 generic.go:356] "Generic (PLEG): container finished" podID="5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0" containerID="4b53275ec37b9d23142c4c2503ebe6ea2a5b3681d3fce3fa26d203da57b98467" exitCode=0
Sep 09 12:27:05.629976 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:05.629625    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml" event={"ID":"5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0","Type":"ContainerDied","Data":"4b53275ec37b9d23142c4c2503ebe6ea2a5b3681d3fce3fa26d203da57b98467"}
Sep 09 12:27:06.771794 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:06.771770    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"
Sep 09 12:27:06.940439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:06.940357    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0-maas-api-service-ca\") pod \"5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0\" (UID: \"5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0\") "
Sep 09 12:27:06.940872 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:06.940846    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0" (UID: "5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:27:07.042432 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:07.042392    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:07.642585 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:07.642555    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"
Sep 09 12:27:07.642771 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:07.642594    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml" event={"ID":"5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0","Type":"ContainerDied","Data":"10781367cdab34637daf644160fe566d7dfa9f7a5745c64679803cd6a5d2941a"}
Sep 09 12:27:07.642771 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:07.642640    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10781367cdab34637daf644160fe566d7dfa9f7a5745c64679803cd6a5d2941a"
Sep 09 12:27:07.872141 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:07.872104    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"]
Sep 09 12:27:07.877420 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:07.877392    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-a1e1f1-tg454"]
Sep 09 12:27:07.899543 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:07.899469    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfc68881-3a91-4956-8da1-09e922e9f574" path="/var/lib/kubelet/pods/bfc68881-3a91-4956-8da1-09e922e9f574/volumes"
Sep 09 12:27:10.844452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:10.844416    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"]
Sep 09 12:27:10.846589 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:10.846566    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e3e6ebae-4vhml"]
Sep 09 12:27:11.899680 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:11.899646    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0" path="/var/lib/kubelet/pods/5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0/volumes"
Sep 09 12:27:21.691292 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:21.691202    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"]
Sep 09 12:27:21.691833 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:21.691469    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy" containerID="cri-o://65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f" gracePeriod=30
Sep 09 12:27:24.651572 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:24.651530    2585 patch_prober.go:28] interesting pod/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 12:27:24.651963 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:24.651601    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:27:26.958370 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:26.958340    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:27:27.058937 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.058902    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istiod-ca-cert\") pod \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") "
Sep 09 12:27:27.059217 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.058956    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-podinfo\") pod \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") "
Sep 09 12:27:27.059217 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059035    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-envoy\") pod \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") "
Sep 09 12:27:27.059217 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059063    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-socket\") pod \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") "
Sep 09 12:27:27.059217 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059132    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-credential-socket\") pod \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") "
Sep 09 12:27:27.059217 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059158    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-certs\") pod \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") "
Sep 09 12:27:27.059561 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059290    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" (UID: "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:27:27.059561 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059373    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" (UID: "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:27.059561 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059377    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-token\") pod \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") "
Sep 09 12:27:27.059561 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059391    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" (UID: "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:27.059561 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059412    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-data\") pod \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") "
Sep 09 12:27:27.059561 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059442    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n74cg\" (UniqueName: \"kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-kube-api-access-n74cg\") pod \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\" (UID: \"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1\") "
Sep 09 12:27:27.059561 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059538    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" (UID: "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:27.060334 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.059654    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" (UID: "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:27.060334 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.060139    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:27.060334 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.060160    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:27.060334 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.060174    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:27.060334 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.060204    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:27.060334 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.060218    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:27.061866 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.061834    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" (UID: "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:27:27.061866 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.061838    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" (UID: "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:27.062164 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.062146    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" (UID: "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:27:27.062431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.062408    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-kube-api-access-n74cg" (OuterVolumeSpecName: "kube-api-access-n74cg") pod "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" (UID: "f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1"). InnerVolumeSpecName "kube-api-access-n74cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:27:27.161320 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.161275    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:27.161320 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.161306    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:27.161320 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.161315    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:27.161320 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.161323    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n74cg\" (UniqueName: \"kubernetes.io/projected/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1-kube-api-access-n74cg\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:27:27.767119 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.767085    2585 generic.go:356] "Generic (PLEG): container finished" podID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerID="65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f" exitCode=0
Sep 09 12:27:27.767290 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.767134    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" event={"ID":"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1","Type":"ContainerDied","Data":"65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f"}
Sep 09 12:27:27.767290 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.767146    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"
Sep 09 12:27:27.767290 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.767176    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n" event={"ID":"f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1","Type":"ContainerDied","Data":"29cb9503406c9d53897b4818db3ba455afd5e952867e626a59ded6dc5fb4bec5"}
Sep 09 12:27:27.767290 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.767197    2585 scope.go:117] "RemoveContainer" containerID="65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f"
Sep 09 12:27:27.777149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.777130    2585 scope.go:117] "RemoveContainer" containerID="a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573"
Sep 09 12:27:27.785733 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.785711    2585 scope.go:117] "RemoveContainer" containerID="65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f"
Sep 09 12:27:27.786004 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:27:27.785987    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f\": container with ID starting with 65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f not found: ID does not exist" containerID="65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f"
Sep 09 12:27:27.786057 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.786014    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f"} err="failed to get container status \"65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f\": rpc error: code = NotFound desc = could not find container \"65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f\": container with ID starting with 65b40a09b3574418a9682840c368e77ee07a2138d6a9d8da1b377e05aeb70a7f not found: ID does not exist"
Sep 09 12:27:27.786057 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.786031    2585 scope.go:117] "RemoveContainer" containerID="a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573"
Sep 09 12:27:27.786299 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:27:27.786281    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573\": container with ID starting with a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573 not found: ID does not exist" containerID="a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573"
Sep 09 12:27:27.786357 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.786304    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573"} err="failed to get container status \"a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573\": rpc error: code = NotFound desc = could not find container \"a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573\": container with ID starting with a6a1adf048a5af9a49d6f8920e9697c738a58e56fc996fc385222237babef573 not found: ID does not exist"
Sep 09 12:27:27.789880 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.789858    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"]
Sep 09 12:27:27.796922 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.796896    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-a1e1f1-openshift-default-7df7f7b6b4-26v6n"]
Sep 09 12:27:27.901575 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:27.901544    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" path="/var/lib/kubelet/pods/f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1/volumes"
Sep 09 12:27:45.599538 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.599499    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"]
Sep 09 12:27:45.600241 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.600220    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy"
Sep 09 12:27:45.600310 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.600247    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy"
Sep 09 12:27:45.600310 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.600281    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0" containerName="revoke-keys"
Sep 09 12:27:45.600310 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.600290    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0" containerName="revoke-keys"
Sep 09 12:27:45.600419 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.600335    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy"
Sep 09 12:27:45.600419 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.600349    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy"
Sep 09 12:27:45.600484 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.600450    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy"
Sep 09 12:27:45.600484 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.600463    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0a9e9f9-eba9-4d4c-99ab-3b7525bf27e1" containerName="istio-proxy"
Sep 09 12:27:45.600484 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.600477    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a03c6fd-a1a2-450c-af5c-1f0e0b0e3aa0" containerName="revoke-keys"
Sep 09 12:27:45.605404 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.605384    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.613147 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.609989    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-c554db\"/\"openshift-service-ca.crt\""
Sep 09 12:27:45.613147 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.610023    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-c554db\"/\"kube-root-ca.crt\""
Sep 09 12:27:45.613147 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.610086    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-c554db\"/\"default-dockercfg-lwttb\""
Sep 09 12:27:45.613147 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.610645    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-c554db\"/\"premium-w0-c554db-kserve-self-signed-certs\""
Sep 09 12:27:45.613638 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.613611    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"]
Sep 09 12:27:45.639853 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.639822    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-model-cache\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.640002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.639874    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-home\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.640002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.639901    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p29sx\" (UniqueName: \"kubernetes.io/projected/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-kube-api-access-p29sx\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.640002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.639963    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tmp-dir\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.640002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.639988    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tls-certs\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.640204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.640012    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-dshm\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.741509 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.741461    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-model-cache\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.741763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.741530    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-home\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.741763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.741561    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p29sx\" (UniqueName: \"kubernetes.io/projected/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-kube-api-access-p29sx\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.741763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.741607    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tmp-dir\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.741763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.741634    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tls-certs\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.741763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.741671    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-dshm\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.742059 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.742003    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-home\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.742059 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.742003    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-model-cache\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.742059 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.742046    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tmp-dir\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.744116 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.744096    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-dshm\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.744371 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.744354    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tls-certs\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.750040 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.750020    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p29sx\" (UniqueName: \"kubernetes.io/projected/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-kube-api-access-p29sx\") pod \"premium-w0-c554db-kserve-6cbdc7bd5-fqs8f\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") " pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:45.921569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:45.921492    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:46.196989 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:46.196962    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"]
Sep 09 12:27:46.198665 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:27:46.198637    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66c97af3_b0ff_4dfb_9c86_32a5b3a2dcf6.slice/crio-6d832a3fadfe0153754f54e5540c023fbf983391af2b41ec6c0c5748b926e789 WatchSource:0}: Error finding container 6d832a3fadfe0153754f54e5540c023fbf983391af2b41ec6c0c5748b926e789: Status 404 returned error can't find the container with id 6d832a3fadfe0153754f54e5540c023fbf983391af2b41ec6c0c5748b926e789
Sep 09 12:27:46.878941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:46.878911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f" event={"ID":"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6","Type":"ContainerStarted","Data":"feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7"}
Sep 09 12:27:46.879369 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:46.878950    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f" event={"ID":"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6","Type":"ContainerStarted","Data":"6d832a3fadfe0153754f54e5540c023fbf983391af2b41ec6c0c5748b926e789"}
Sep 09 12:27:46.897612 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:46.897562    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f" podStartSLOduration=1.897548738 podStartE2EDuration="1.897548738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:27:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:27:46.894945898 +0000 UTC m=+1011.536565104" watchObservedRunningTime="2026-09-09 12:27:46.897548738 +0000 UTC m=+1011.539167942"
Sep 09 12:27:55.922105 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:55.922071    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:55.922105 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:55.922120    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:55.936000 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:55.935973    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:27:55.947193 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:27:55.947166    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:28:15.004202 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.004171    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"]
Sep 09 12:28:15.009048 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.009032    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.012505 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.012481    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2a142a\"/\"kube-root-ca.crt\""
Sep 09 12:28:15.012629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.012511    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2a142a\"/\"default-dockercfg-t6nx2\""
Sep 09 12:28:15.012629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.012575    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2a142a\"/\"test-model-2a142a-kserve-self-signed-certs\""
Sep 09 12:28:15.012629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.012575    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2a142a\"/\"openshift-service-ca.crt\""
Sep 09 12:28:15.018972 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.018949    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"]
Sep 09 12:28:15.151561 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.151520    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-tmp-dir\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.151772 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.151571    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-dshm\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.151772 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.151653    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xbsr\" (UniqueName: \"kubernetes.io/projected/774222e7-be56-43ab-8877-9c3e8c4e21d7-kube-api-access-8xbsr\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.151772 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.151693    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-model-cache\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.151905 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.151865    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-home\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.151905 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.151899    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/774222e7-be56-43ab-8877-9c3e8c4e21d7-tls-certs\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.253517 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.253481    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-home\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.253750 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.253526    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/774222e7-be56-43ab-8877-9c3e8c4e21d7-tls-certs\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.253750 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.253577    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-tmp-dir\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.253750 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.253600    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-dshm\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.253750 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.253642    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xbsr\" (UniqueName: \"kubernetes.io/projected/774222e7-be56-43ab-8877-9c3e8c4e21d7-kube-api-access-8xbsr\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.253750 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.253677    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-model-cache\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.254042 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.253984    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-home\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.254042 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.254026    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-tmp-dir\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.254245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.254177    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-model-cache\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.256680 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.256648    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-dshm\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.256951 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.256936    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/774222e7-be56-43ab-8877-9c3e8c4e21d7-tls-certs\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.263444 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.263424    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xbsr\" (UniqueName: \"kubernetes.io/projected/774222e7-be56-43ab-8877-9c3e8c4e21d7-kube-api-access-8xbsr\") pod \"test-model-2a142a-kserve-cb94775ff-whgbz\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") " pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.318574 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.318544    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:15.598049 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:15.598023    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"]
Sep 09 12:28:15.600201 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:28:15.600176    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod774222e7_be56_43ab_8877_9c3e8c4e21d7.slice/crio-914cbf3ab355e02e1197fc53a3b19b30f8accf01a6a3ad758fe651e747487a05 WatchSource:0}: Error finding container 914cbf3ab355e02e1197fc53a3b19b30f8accf01a6a3ad758fe651e747487a05: Status 404 returned error can't find the container with id 914cbf3ab355e02e1197fc53a3b19b30f8accf01a6a3ad758fe651e747487a05
Sep 09 12:28:16.042851 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:16.042818    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz" event={"ID":"774222e7-be56-43ab-8877-9c3e8c4e21d7","Type":"ContainerStarted","Data":"b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741"}
Sep 09 12:28:16.043238 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:16.042864    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz" event={"ID":"774222e7-be56-43ab-8877-9c3e8c4e21d7","Type":"ContainerStarted","Data":"914cbf3ab355e02e1197fc53a3b19b30f8accf01a6a3ad758fe651e747487a05"}
Sep 09 12:28:16.065668 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:16.065619    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz" podStartSLOduration=2.065599942 podStartE2EDuration="2.065599942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:16.063913511 +0000 UTC m=+1040.705532717" watchObservedRunningTime="2026-09-09 12:28:16.065599942 +0000 UTC m=+1040.707219146"
Sep 09 12:28:25.319573 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:25.319531    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:25.320019 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:25.319626    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:25.332606 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:25.332581    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:28:26.114889 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:28:26.114849    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:29:00.785193 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.785160    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"]
Sep 09 12:29:00.788775 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.788754    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.791376 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.791353    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-c554db\"/\"distinct-w0-c554db-kserve-self-signed-certs\""
Sep 09 12:29:00.802358 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.802331    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"]
Sep 09 12:29:00.816220 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.816195    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-home\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.816345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.816227    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-dshm\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.816396 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.816354    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tmp-dir\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.816445 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.816419    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tls-certs\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.816491 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.816471    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-model-cache\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.816536 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.816514    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrqbp\" (UniqueName: \"kubernetes.io/projected/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-kube-api-access-xrqbp\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.917883 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.917824    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tls-certs\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.918094 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.917900    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-model-cache\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.918094 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.917933    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrqbp\" (UniqueName: \"kubernetes.io/projected/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-kube-api-access-xrqbp\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.918340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.918313    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-model-cache\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.918340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.917986    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-home\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.918547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.918353    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-home\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.918547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.918376    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-dshm\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.918547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.918483    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tmp-dir\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.919013 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.918989    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tmp-dir\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.920628 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.920603    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tls-certs\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.920716 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.920601    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-dshm\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:00.928540 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:00.928517    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrqbp\" (UniqueName: \"kubernetes.io/projected/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-kube-api-access-xrqbp\") pod \"distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:01.099358 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:01.099273    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:01.380447 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:01.380413    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"]
Sep 09 12:29:01.382279 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:29:01.382252    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc90e5774_cb91_45a7_ae31_3a1a8c2a48e6.slice/crio-c837f85b0f4f4ac3cced713b09f6925494e9f03b6b9103baafa91a9dcc73c585 WatchSource:0}: Error finding container c837f85b0f4f4ac3cced713b09f6925494e9f03b6b9103baafa91a9dcc73c585: Status 404 returned error can't find the container with id c837f85b0f4f4ac3cced713b09f6925494e9f03b6b9103baafa91a9dcc73c585
Sep 09 12:29:02.317905 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:02.317874    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5" event={"ID":"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6","Type":"ContainerStarted","Data":"37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15"}
Sep 09 12:29:02.317905 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:02.317911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5" event={"ID":"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6","Type":"ContainerStarted","Data":"c837f85b0f4f4ac3cced713b09f6925494e9f03b6b9103baafa91a9dcc73c585"}
Sep 09 12:29:02.338440 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:02.338380    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5" podStartSLOduration=2.338363123 podStartE2EDuration="2.338363123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:29:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:29:02.336646681 +0000 UTC m=+1086.978265889" watchObservedRunningTime="2026-09-09 12:29:02.338363123 +0000 UTC m=+1086.979982324"
Sep 09 12:29:05.833713 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:05.833664    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"]
Sep 09 12:29:05.836898 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:05.836883    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"
Sep 09 12:29:05.839293 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:05.839271    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:29:05.839411 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:05.839276    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:29:05.846004 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:05.845441    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"]
Sep 09 12:29:05.972798 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:05.972757    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b340743d-537f-435d-892d-20f76d6b6971-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp\" (UID: \"b340743d-537f-435d-892d-20f76d6b6971\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"
Sep 09 12:29:06.073456 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:06.073423    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b340743d-537f-435d-892d-20f76d6b6971-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp\" (UID: \"b340743d-537f-435d-892d-20f76d6b6971\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"
Sep 09 12:29:06.074126 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:06.074104    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b340743d-537f-435d-892d-20f76d6b6971-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp\" (UID: \"b340743d-537f-435d-892d-20f76d6b6971\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"
Sep 09 12:29:06.150502 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:06.150369    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"
Sep 09 12:29:06.426259 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:06.426235    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"]
Sep 09 12:29:06.428352 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:29:06.428325    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb340743d_537f_435d_892d_20f76d6b6971.slice/crio-cacb6f0b45cf925021b5085d503cf91770b0fef612bc62b2272a5501cb64f433 WatchSource:0}: Error finding container cacb6f0b45cf925021b5085d503cf91770b0fef612bc62b2272a5501cb64f433: Status 404 returned error can't find the container with id cacb6f0b45cf925021b5085d503cf91770b0fef612bc62b2272a5501cb64f433
Sep 09 12:29:07.351576 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:07.351545    2585 generic.go:356] "Generic (PLEG): container finished" podID="b340743d-537f-435d-892d-20f76d6b6971" containerID="5adf35c769a18cc304314658b127074571a506fa92dfd7fbcd55b525cff67abc" exitCode=0
Sep 09 12:29:07.352073 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:07.351634    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp" event={"ID":"b340743d-537f-435d-892d-20f76d6b6971","Type":"ContainerDied","Data":"5adf35c769a18cc304314658b127074571a506fa92dfd7fbcd55b525cff67abc"}
Sep 09 12:29:07.352073 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:07.351682    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp" event={"ID":"b340743d-537f-435d-892d-20f76d6b6971","Type":"ContainerStarted","Data":"cacb6f0b45cf925021b5085d503cf91770b0fef612bc62b2272a5501cb64f433"}
Sep 09 12:29:08.497209 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:08.497188    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"
Sep 09 12:29:08.599232 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:08.599198    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b340743d-537f-435d-892d-20f76d6b6971-maas-api-service-ca\") pod \"b340743d-537f-435d-892d-20f76d6b6971\" (UID: \"b340743d-537f-435d-892d-20f76d6b6971\") "
Sep 09 12:29:08.599601 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:08.599575    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b340743d-537f-435d-892d-20f76d6b6971-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b340743d-537f-435d-892d-20f76d6b6971" (UID: "b340743d-537f-435d-892d-20f76d6b6971"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:29:08.700415 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:08.700334    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b340743d-537f-435d-892d-20f76d6b6971-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:29:09.365577 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:09.365541    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp" event={"ID":"b340743d-537f-435d-892d-20f76d6b6971","Type":"ContainerDied","Data":"cacb6f0b45cf925021b5085d503cf91770b0fef612bc62b2272a5501cb64f433"}
Sep 09 12:29:09.365577 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:09.365576    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"
Sep 09 12:29:09.365813 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:09.365594    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cacb6f0b45cf925021b5085d503cf91770b0fef612bc62b2272a5501cb64f433"
Sep 09 12:29:11.100106 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:11.100077    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:11.100520 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:11.100328    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:11.113045 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:11.113018    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:11.388822 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:11.388734    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:29:16.687857 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:16.687817    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"]
Sep 09 12:29:16.690924 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:16.690889    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b402ba-hlpdp"]
Sep 09 12:29:17.900066 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:17.900024    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b340743d-537f-435d-892d-20f76d6b6971" path="/var/lib/kubelet/pods/b340743d-537f-435d-892d-20f76d6b6971/volumes"
Sep 09 12:29:24.748489 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.748449    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"]
Sep 09 12:29:24.749140 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.749118    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b340743d-537f-435d-892d-20f76d6b6971" containerName="revoke-keys"
Sep 09 12:29:24.749140 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.749139    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b340743d-537f-435d-892d-20f76d6b6971" containerName="revoke-keys"
Sep 09 12:29:24.749290 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.749278    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b340743d-537f-435d-892d-20f76d6b6971" containerName="revoke-keys"
Sep 09 12:29:24.752759 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.752736    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.755197 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.755167    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-db67bc-openshift-default-dockercfg-tklcn\""
Sep 09 12:29:24.764048 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.764024    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"]
Sep 09 12:29:24.769735 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.769687    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-credential-socket\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.769844 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.769752    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-socket\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.769844 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.769780    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-data\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.769844 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.769801    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-certs\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.769984 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.769885    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-podinfo\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.769984 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.769920    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd8f6a5-238b-4be8-837d-caf16221558d-istiod-ca-cert\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.769984 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.769956    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-envoy\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.769984 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.769975    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-token\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.770144 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.770071    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvhk8\" (UniqueName: \"kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-kube-api-access-gvhk8\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.870805 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.870757    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd8f6a5-238b-4be8-837d-caf16221558d-istiod-ca-cert\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.870980 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.870819    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-envoy\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.870980 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.870843    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-token\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.870980 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.870884    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvhk8\" (UniqueName: \"kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-kube-api-access-gvhk8\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.871134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.870982    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-credential-socket\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.871134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.871006    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-socket\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.871134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.871044    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-data\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.871134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.871074    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-certs\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.871542 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.871147    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-podinfo\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.871542 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.871409    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-credential-socket\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.871542 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.871460    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-socket\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.871542 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.871487    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-data\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.871542 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.871508    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-certs\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.871779 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.871620    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd8f6a5-238b-4be8-837d-caf16221558d-istiod-ca-cert\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.873734 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.873686    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-envoy\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.873923 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.873907    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-podinfo\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.882326 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.882302    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-token\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:24.882437 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:24.882377    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvhk8\" (UniqueName: \"kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-kube-api-access-gvhk8\") pod \"e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") " pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:25.066011 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:25.065980    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:25.347848 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:25.347820    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"]
Sep 09 12:29:25.353517 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:25.353473    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:29:25.353635 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:25.353573    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:29:25.353635 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:25.353618    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:29:25.464724 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:25.464657    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv" event={"ID":"9cd8f6a5-238b-4be8-837d-caf16221558d","Type":"ContainerStarted","Data":"7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec"}
Sep 09 12:29:25.464893 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:25.464758    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv" event={"ID":"9cd8f6a5-238b-4be8-837d-caf16221558d","Type":"ContainerStarted","Data":"a4cc418c02fc62cf8e64065acd023c6ffbb59a242304e58b84c1945f662490ff"}
Sep 09 12:29:25.485337 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:25.484829    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv" podStartSLOduration=1.484808815 podStartE2EDuration="1.484808815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:29:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:29:25.484083233 +0000 UTC m=+1110.125702438" watchObservedRunningTime="2026-09-09 12:29:25.484808815 +0000 UTC m=+1110.126428020"
Sep 09 12:29:26.066541 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:26.066506    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:26.072316 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:26.072280    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:26.471744 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:26.471646    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:29:26.472687 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:29:26.472668    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:30:00.158464 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.158424    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh"]
Sep 09 12:30:00.162373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.162356    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh"
Sep 09 12:30:00.164737 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.164721    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:30:00.172041 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.172012    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815950-gmqlp"]
Sep 09 12:30:00.174942 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.174912    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh"
Sep 09 12:30:00.178275 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.178053    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh"]
Sep 09 12:30:00.178275 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.178161    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815950-gmqlp"
Sep 09 12:30:00.191309 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.187917    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815950-gmqlp"]
Sep 09 12:30:00.216693 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.216663    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815950-gmqlp"
Sep 09 12:30:00.463050 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.463019    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh"]
Sep 09 12:30:00.465040 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:30:00.465000    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode083115b_be70_4df7_a16e_b7846ef315ef.slice/crio-5aa1e7665f50acf8813b196c34fd66463ec0d6751127b64024b63855d698610a WatchSource:0}: Error finding container 5aa1e7665f50acf8813b196c34fd66463ec0d6751127b64024b63855d698610a: Status 404 returned error can't find the container with id 5aa1e7665f50acf8813b196c34fd66463ec0d6751127b64024b63855d698610a
Sep 09 12:30:00.505776 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.505754    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815950-gmqlp"]
Sep 09 12:30:00.508672 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:30:00.508640    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod647ff576_cf6d_4936_a267_4f6611c55ed8.slice/crio-918e7d4f16151bfd3d7f78325e8be26d43e46c1a8684a142700eb9e9d60c8454 WatchSource:0}: Error finding container 918e7d4f16151bfd3d7f78325e8be26d43e46c1a8684a142700eb9e9d60c8454: Status 404 returned error can't find the container with id 918e7d4f16151bfd3d7f78325e8be26d43e46c1a8684a142700eb9e9d60c8454
Sep 09 12:30:00.670874 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.670832    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh" event={"ID":"e083115b-be70-4df7-a16e-b7846ef315ef","Type":"ContainerStarted","Data":"5aa1e7665f50acf8813b196c34fd66463ec0d6751127b64024b63855d698610a"}
Sep 09 12:30:00.672519 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:00.672478    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815950-gmqlp" event={"ID":"647ff576-cf6d-4936-a267-4f6611c55ed8","Type":"ContainerStarted","Data":"918e7d4f16151bfd3d7f78325e8be26d43e46c1a8684a142700eb9e9d60c8454"}
Sep 09 12:30:01.684607 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:01.684572    2585 generic.go:356] "Generic (PLEG): container finished" podID="e083115b-be70-4df7-a16e-b7846ef315ef" containerID="02333559a18053318b139de6a817978018ea1bc4916688133e26600123cc2e7e" exitCode=0
Sep 09 12:30:01.685077 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:01.684659    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh" event={"ID":"e083115b-be70-4df7-a16e-b7846ef315ef","Type":"ContainerDied","Data":"02333559a18053318b139de6a817978018ea1bc4916688133e26600123cc2e7e"}
Sep 09 12:30:01.686213 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:01.686191    2585 generic.go:356] "Generic (PLEG): container finished" podID="647ff576-cf6d-4936-a267-4f6611c55ed8" containerID="0ffd9465fb5a51759056d959a991178c59a538c5125db634bab44a33da1ed612" exitCode=0
Sep 09 12:30:01.686349 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:01.686268    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815950-gmqlp" event={"ID":"647ff576-cf6d-4936-a267-4f6611c55ed8","Type":"ContainerDied","Data":"0ffd9465fb5a51759056d959a991178c59a538c5125db634bab44a33da1ed612"}
Sep 09 12:30:02.865901 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:02.865873    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815950-gmqlp"
Sep 09 12:30:02.869987 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:02.869964    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh"
Sep 09 12:30:03.701868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:03.701841    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815950-gmqlp"
Sep 09 12:30:03.702115 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:03.701877    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815950-gmqlp" event={"ID":"647ff576-cf6d-4936-a267-4f6611c55ed8","Type":"ContainerDied","Data":"918e7d4f16151bfd3d7f78325e8be26d43e46c1a8684a142700eb9e9d60c8454"}
Sep 09 12:30:03.702115 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:03.701926    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="918e7d4f16151bfd3d7f78325e8be26d43e46c1a8684a142700eb9e9d60c8454"
Sep 09 12:30:03.703484 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:03.703447    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh" event={"ID":"e083115b-be70-4df7-a16e-b7846ef315ef","Type":"ContainerDied","Data":"5aa1e7665f50acf8813b196c34fd66463ec0d6751127b64024b63855d698610a"}
Sep 09 12:30:03.703621 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:03.703501    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5aa1e7665f50acf8813b196c34fd66463ec0d6751127b64024b63855d698610a"
Sep 09 12:30:03.703621 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:03.703462    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh"
Sep 09 12:30:45.780691 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.780653    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"]
Sep 09 12:30:45.781139 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.781107    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="647ff576-cf6d-4936-a267-4f6611c55ed8" containerName="cleanup"
Sep 09 12:30:45.781139 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.781119    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="647ff576-cf6d-4936-a267-4f6611c55ed8" containerName="cleanup"
Sep 09 12:30:45.781216 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.781141    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e083115b-be70-4df7-a16e-b7846ef315ef" containerName="cleanup"
Sep 09 12:30:45.781216 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.781147    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e083115b-be70-4df7-a16e-b7846ef315ef" containerName="cleanup"
Sep 09 12:30:45.781278 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.781224    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e083115b-be70-4df7-a16e-b7846ef315ef" containerName="cleanup"
Sep 09 12:30:45.781278 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.781235    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="647ff576-cf6d-4936-a267-4f6611c55ed8" containerName="cleanup"
Sep 09 12:30:45.784787 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.784768    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"
Sep 09 12:30:45.787037 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.787018    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:30:45.787176 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.787018    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:30:45.795946 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.795918    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"]
Sep 09 12:30:45.820199 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.820171    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a3cbd8-65bd-4f3a-85ef-e64dbe793131-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6\" (UID: \"b4a3cbd8-65bd-4f3a-85ef-e64dbe793131\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"
Sep 09 12:30:45.920965 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.920936    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a3cbd8-65bd-4f3a-85ef-e64dbe793131-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6\" (UID: \"b4a3cbd8-65bd-4f3a-85ef-e64dbe793131\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"
Sep 09 12:30:45.921543 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:45.921523    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a3cbd8-65bd-4f3a-85ef-e64dbe793131-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6\" (UID: \"b4a3cbd8-65bd-4f3a-85ef-e64dbe793131\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"
Sep 09 12:30:46.095630 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:46.095551    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"
Sep 09 12:30:46.370687 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:46.370611    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"]
Sep 09 12:30:46.374232 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:30:46.374198    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a3cbd8_65bd_4f3a_85ef_e64dbe793131.slice/crio-66e1683cfa3ac9459576f14a5cf06aa4860ed10c7423cfe43dd5936b8913e258 WatchSource:0}: Error finding container 66e1683cfa3ac9459576f14a5cf06aa4860ed10c7423cfe43dd5936b8913e258: Status 404 returned error can't find the container with id 66e1683cfa3ac9459576f14a5cf06aa4860ed10c7423cfe43dd5936b8913e258
Sep 09 12:30:46.970942 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:46.970849    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6" event={"ID":"b4a3cbd8-65bd-4f3a-85ef-e64dbe793131","Type":"ContainerStarted","Data":"02ad02f9c34c7b5170f10d8f69ccddf6f970e9e66fa45249e20d1f996a2b3bfd"}
Sep 09 12:30:46.970942 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:46.970905    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6" event={"ID":"b4a3cbd8-65bd-4f3a-85ef-e64dbe793131","Type":"ContainerStarted","Data":"66e1683cfa3ac9459576f14a5cf06aa4860ed10c7423cfe43dd5936b8913e258"}
Sep 09 12:30:47.018617 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:47.018568    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6" podStartSLOduration=2.018547883 podStartE2EDuration="2.018547883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:30:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:30:47.017860373 +0000 UTC m=+1191.659479577" watchObservedRunningTime="2026-09-09 12:30:47.018547883 +0000 UTC m=+1191.660167087"
Sep 09 12:30:47.978043 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:47.978004    2585 generic.go:356] "Generic (PLEG): container finished" podID="b4a3cbd8-65bd-4f3a-85ef-e64dbe793131" containerID="02ad02f9c34c7b5170f10d8f69ccddf6f970e9e66fa45249e20d1f996a2b3bfd" exitCode=0
Sep 09 12:30:47.978428 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:47.978099    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6" event={"ID":"b4a3cbd8-65bd-4f3a-85ef-e64dbe793131","Type":"ContainerDied","Data":"02ad02f9c34c7b5170f10d8f69ccddf6f970e9e66fa45249e20d1f996a2b3bfd"}
Sep 09 12:30:49.124778 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:49.124756    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"
Sep 09 12:30:49.260374 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:49.260342    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a3cbd8-65bd-4f3a-85ef-e64dbe793131-maas-api-service-ca\") pod \"b4a3cbd8-65bd-4f3a-85ef-e64dbe793131\" (UID: \"b4a3cbd8-65bd-4f3a-85ef-e64dbe793131\") "
Sep 09 12:30:49.260675 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:49.260654    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4a3cbd8-65bd-4f3a-85ef-e64dbe793131-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b4a3cbd8-65bd-4f3a-85ef-e64dbe793131" (UID: "b4a3cbd8-65bd-4f3a-85ef-e64dbe793131"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:30:49.361943 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:49.361906    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a3cbd8-65bd-4f3a-85ef-e64dbe793131-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:30:49.990961 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:49.990933    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"
Sep 09 12:30:49.991147 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:49.990976    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6" event={"ID":"b4a3cbd8-65bd-4f3a-85ef-e64dbe793131","Type":"ContainerDied","Data":"66e1683cfa3ac9459576f14a5cf06aa4860ed10c7423cfe43dd5936b8913e258"}
Sep 09 12:30:49.991147 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:49.991025    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66e1683cfa3ac9459576f14a5cf06aa4860ed10c7423cfe43dd5936b8913e258"
Sep 09 12:30:51.286505 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.286466    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"]
Sep 09 12:30:51.287151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.287131    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a3cbd8-65bd-4f3a-85ef-e64dbe793131" containerName="revoke-keys"
Sep 09 12:30:51.287151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.287153    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a3cbd8-65bd-4f3a-85ef-e64dbe793131" containerName="revoke-keys"
Sep 09 12:30:51.287249 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.287219    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a3cbd8-65bd-4f3a-85ef-e64dbe793131" containerName="revoke-keys"
Sep 09 12:30:51.290254 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.290239    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.292469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.292451    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-c554db\"/\"distinct-2-w0-c554db-kserve-self-signed-certs\""
Sep 09 12:30:51.298921 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.298898    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"]
Sep 09 12:30:51.383889 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.383847    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-model-cache\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.384096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.383928    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-dshm\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.384096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.383974    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-tmp-dir\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.384096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.384071    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-home\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.384253 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.384104    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/615e2500-2db0-4d7c-9a69-69348178f6a6-tls-certs\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.384253 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.384183    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5vff\" (UniqueName: \"kubernetes.io/projected/615e2500-2db0-4d7c-9a69-69348178f6a6-kube-api-access-w5vff\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.485679 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.485638    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-home\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.485679 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.485680    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/615e2500-2db0-4d7c-9a69-69348178f6a6-tls-certs\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.485934 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.485782    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5vff\" (UniqueName: \"kubernetes.io/projected/615e2500-2db0-4d7c-9a69-69348178f6a6-kube-api-access-w5vff\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.485934 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.485837    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-model-cache\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.485934 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.485883    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-dshm\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.486059 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.485931    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-tmp-dir\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.486144 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.486126    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-home\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.486213 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.486182    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-model-cache\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.486340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.486276    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-tmp-dir\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.488257 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.488236    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-dshm\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.488452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.488434    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/615e2500-2db0-4d7c-9a69-69348178f6a6-tls-certs\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.495878 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.495855    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5vff\" (UniqueName: \"kubernetes.io/projected/615e2500-2db0-4d7c-9a69-69348178f6a6-kube-api-access-w5vff\") pod \"distinct-2-w0-c554db-kserve-f78699c9f-p4b67\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") " pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.599477 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.599388    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:30:51.879427 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:51.879402    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"]
Sep 09 12:30:51.881511 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:30:51.881485    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod615e2500_2db0_4d7c_9a69_69348178f6a6.slice/crio-589ec58661d80988f39eb8d1db08e836d8e74dd8b1503c041f5df13231cf91ea WatchSource:0}: Error finding container 589ec58661d80988f39eb8d1db08e836d8e74dd8b1503c041f5df13231cf91ea: Status 404 returned error can't find the container with id 589ec58661d80988f39eb8d1db08e836d8e74dd8b1503c041f5df13231cf91ea
Sep 09 12:30:52.005218 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:52.005184    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67" event={"ID":"615e2500-2db0-4d7c-9a69-69348178f6a6","Type":"ContainerStarted","Data":"1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9"}
Sep 09 12:30:52.005390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:52.005226    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67" event={"ID":"615e2500-2db0-4d7c-9a69-69348178f6a6","Type":"ContainerStarted","Data":"589ec58661d80988f39eb8d1db08e836d8e74dd8b1503c041f5df13231cf91ea"}
Sep 09 12:30:52.028626 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:52.028576    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67" podStartSLOduration=1.028560865 podStartE2EDuration="1.028560865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:30:52.028241918 +0000 UTC m=+1196.669861142" watchObservedRunningTime="2026-09-09 12:30:52.028560865 +0000 UTC m=+1196.670180069"
Sep 09 12:30:55.106786 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:55.106753    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"]
Sep 09 12:30:57.705195 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:57.705151    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh"]
Sep 09 12:30:57.711944 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:57.711915    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e49066-29815950-6mlfh"]
Sep 09 12:30:57.900118 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:57.900081    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e083115b-be70-4df7-a16e-b7846ef315ef" path="/var/lib/kubelet/pods/e083115b-be70-4df7-a16e-b7846ef315ef/volumes"
Sep 09 12:30:58.195435 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:58.195390    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"]
Sep 09 12:30:58.203039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:58.203008    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e49066-vrtm6"]
Sep 09 12:30:59.900686 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:30:59.900648    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4a3cbd8-65bd-4f3a-85ef-e64dbe793131" path="/var/lib/kubelet/pods/b4a3cbd8-65bd-4f3a-85ef-e64dbe793131/volumes"
Sep 09 12:31:01.600410 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:01.600371    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:31:01.600902 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:01.600463    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:31:01.613244 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:01.613213    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:31:02.077718 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:02.077673    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:31:10.126856 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.126806    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx" podUID="364eb190-c9e4-4f34-8b8f-efd15708e5f6" containerName="main" containerID="cri-o://ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d" gracePeriod=45
Sep 09 12:31:10.392308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.392238    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:31:10.492043 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492001    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-dshm\") pod \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") "
Sep 09 12:31:10.492216 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492082    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tls-certs\") pod \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") "
Sep 09 12:31:10.492216 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492119    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-model-cache\") pod \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") "
Sep 09 12:31:10.492216 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492147    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngzzz\" (UniqueName: \"kubernetes.io/projected/364eb190-c9e4-4f34-8b8f-efd15708e5f6-kube-api-access-ngzzz\") pod \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") "
Sep 09 12:31:10.492216 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492165    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-home\") pod \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") "
Sep 09 12:31:10.492216 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492189    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tmp-dir\") pod \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\" (UID: \"364eb190-c9e4-4f34-8b8f-efd15708e5f6\") "
Sep 09 12:31:10.492516 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492468    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "364eb190-c9e4-4f34-8b8f-efd15708e5f6" (UID: "364eb190-c9e4-4f34-8b8f-efd15708e5f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:10.492516 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492480    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-home" (OuterVolumeSpecName: "home") pod "364eb190-c9e4-4f34-8b8f-efd15708e5f6" (UID: "364eb190-c9e4-4f34-8b8f-efd15708e5f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:10.492690 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492671    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "364eb190-c9e4-4f34-8b8f-efd15708e5f6" (UID: "364eb190-c9e4-4f34-8b8f-efd15708e5f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:10.492779 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492690    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:10.492779 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.492726    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:10.494472 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.494445    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-dshm" (OuterVolumeSpecName: "dshm") pod "364eb190-c9e4-4f34-8b8f-efd15708e5f6" (UID: "364eb190-c9e4-4f34-8b8f-efd15708e5f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:10.494567 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.494481    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/364eb190-c9e4-4f34-8b8f-efd15708e5f6-kube-api-access-ngzzz" (OuterVolumeSpecName: "kube-api-access-ngzzz") pod "364eb190-c9e4-4f34-8b8f-efd15708e5f6" (UID: "364eb190-c9e4-4f34-8b8f-efd15708e5f6"). InnerVolumeSpecName "kube-api-access-ngzzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:31:10.494567 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.494494    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "364eb190-c9e4-4f34-8b8f-efd15708e5f6" (UID: "364eb190-c9e4-4f34-8b8f-efd15708e5f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:31:10.594118 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.594084    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:10.594118 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.594115    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:10.594118 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.594125    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngzzz\" (UniqueName: \"kubernetes.io/projected/364eb190-c9e4-4f34-8b8f-efd15708e5f6-kube-api-access-ngzzz\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:10.594357 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:10.594134    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/364eb190-c9e4-4f34-8b8f-efd15708e5f6-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:11.127114 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:11.127081    2585 generic.go:356] "Generic (PLEG): container finished" podID="364eb190-c9e4-4f34-8b8f-efd15708e5f6" containerID="ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d" exitCode=0
Sep 09 12:31:11.127533 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:11.127126    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx" event={"ID":"364eb190-c9e4-4f34-8b8f-efd15708e5f6","Type":"ContainerDied","Data":"ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d"}
Sep 09 12:31:11.127533 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:11.127140    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"
Sep 09 12:31:11.127533 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:11.127159    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx" event={"ID":"364eb190-c9e4-4f34-8b8f-efd15708e5f6","Type":"ContainerDied","Data":"52fa441c0b08908e3f2df70def6aa8a9241e7b20610ff8d10e6220aad5b55edf"}
Sep 09 12:31:11.127533 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:11.127177    2585 scope.go:117] "RemoveContainer" containerID="ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d"
Sep 09 12:31:11.137738 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:11.137691    2585 scope.go:117] "RemoveContainer" containerID="ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d"
Sep 09 12:31:11.138077 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:31:11.138057    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d\": container with ID starting with ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d not found: ID does not exist" containerID="ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d"
Sep 09 12:31:11.138117 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:11.138088    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d"} err="failed to get container status \"ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d\": rpc error: code = NotFound desc = could not find container \"ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d\": container with ID starting with ed7597bebf9638e7e5e672d4c9b8aa4ef6779e42ec21173f00e08fbb256c627d not found: ID does not exist"
Sep 09 12:31:11.156375 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:11.156340    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"]
Sep 09 12:31:11.161464 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:11.161437    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e49066/test-model-e49066-kserve-6b7c9c7dd9-fjrnx"]
Sep 09 12:31:11.900909 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:11.900872    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="364eb190-c9e4-4f34-8b8f-efd15708e5f6" path="/var/lib/kubelet/pods/364eb190-c9e4-4f34-8b8f-efd15708e5f6/volumes"
Sep 09 12:31:14.623896 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:14.623863    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"]
Sep 09 12:31:14.624355 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:14.624117    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv" podUID="9cd8f6a5-238b-4be8-837d-caf16221558d" containerName="istio-proxy" containerID="cri-o://7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec" gracePeriod=30
Sep 09 12:31:19.875422 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.875393    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:31:19.979615 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979540    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd8f6a5-238b-4be8-837d-caf16221558d-istiod-ca-cert\") pod \"9cd8f6a5-238b-4be8-837d-caf16221558d\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") "
Sep 09 12:31:19.979615 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979582    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-socket\") pod \"9cd8f6a5-238b-4be8-837d-caf16221558d\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") "
Sep 09 12:31:19.979615 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979610    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-certs\") pod \"9cd8f6a5-238b-4be8-837d-caf16221558d\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") "
Sep 09 12:31:19.979920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979628    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-data\") pod \"9cd8f6a5-238b-4be8-837d-caf16221558d\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") "
Sep 09 12:31:19.979920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979642    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-envoy\") pod \"9cd8f6a5-238b-4be8-837d-caf16221558d\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") "
Sep 09 12:31:19.979920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979752    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-token\") pod \"9cd8f6a5-238b-4be8-837d-caf16221558d\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") "
Sep 09 12:31:19.979920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979782    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-credential-socket\") pod \"9cd8f6a5-238b-4be8-837d-caf16221558d\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") "
Sep 09 12:31:19.979920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979801    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvhk8\" (UniqueName: \"kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-kube-api-access-gvhk8\") pod \"9cd8f6a5-238b-4be8-837d-caf16221558d\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") "
Sep 09 12:31:19.979920 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979844    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-podinfo\") pod \"9cd8f6a5-238b-4be8-837d-caf16221558d\" (UID: \"9cd8f6a5-238b-4be8-837d-caf16221558d\") "
Sep 09 12:31:19.980285 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979977    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9cd8f6a5-238b-4be8-837d-caf16221558d" (UID: "9cd8f6a5-238b-4be8-837d-caf16221558d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:19.980285 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.979988    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-data" (OuterVolumeSpecName: "istio-data") pod "9cd8f6a5-238b-4be8-837d-caf16221558d" (UID: "9cd8f6a5-238b-4be8-837d-caf16221558d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:19.980285 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.980011    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9cd8f6a5-238b-4be8-837d-caf16221558d" (UID: "9cd8f6a5-238b-4be8-837d-caf16221558d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:19.980285 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.980039    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cd8f6a5-238b-4be8-837d-caf16221558d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9cd8f6a5-238b-4be8-837d-caf16221558d" (UID: "9cd8f6a5-238b-4be8-837d-caf16221558d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:31:19.980285 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.980170    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9cd8f6a5-238b-4be8-837d-caf16221558d" (UID: "9cd8f6a5-238b-4be8-837d-caf16221558d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:19.980629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.980343    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:19.980629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.980363    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd8f6a5-238b-4be8-837d-caf16221558d-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:19.980629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.980378    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:19.980629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.980387    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:19.980629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.980395    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:19.982173 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.982146    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9cd8f6a5-238b-4be8-837d-caf16221558d" (UID: "9cd8f6a5-238b-4be8-837d-caf16221558d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:19.982282 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.982225    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-kube-api-access-gvhk8" (OuterVolumeSpecName: "kube-api-access-gvhk8") pod "9cd8f6a5-238b-4be8-837d-caf16221558d" (UID: "9cd8f6a5-238b-4be8-837d-caf16221558d"). InnerVolumeSpecName "kube-api-access-gvhk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:31:19.982452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.982432    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-token" (OuterVolumeSpecName: "istio-token") pod "9cd8f6a5-238b-4be8-837d-caf16221558d" (UID: "9cd8f6a5-238b-4be8-837d-caf16221558d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:31:19.982452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:19.982442    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9cd8f6a5-238b-4be8-837d-caf16221558d" (UID: "9cd8f6a5-238b-4be8-837d-caf16221558d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:31:20.081842 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.081806    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvhk8\" (UniqueName: \"kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-kube-api-access-gvhk8\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:20.081842 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.081837    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:20.081842 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.081850    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:20.082100 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.081862    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd8f6a5-238b-4be8-837d-caf16221558d-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:20.186121 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.186085    2585 generic.go:356] "Generic (PLEG): container finished" podID="9cd8f6a5-238b-4be8-837d-caf16221558d" containerID="7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec" exitCode=0
Sep 09 12:31:20.186298 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.186137    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"
Sep 09 12:31:20.186298 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.186167    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv" event={"ID":"9cd8f6a5-238b-4be8-837d-caf16221558d","Type":"ContainerDied","Data":"7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec"}
Sep 09 12:31:20.186298 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.186243    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv" event={"ID":"9cd8f6a5-238b-4be8-837d-caf16221558d","Type":"ContainerDied","Data":"a4cc418c02fc62cf8e64065acd023c6ffbb59a242304e58b84c1945f662490ff"}
Sep 09 12:31:20.186298 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.186274    2585 scope.go:117] "RemoveContainer" containerID="7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec"
Sep 09 12:31:20.196667 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.196647    2585 scope.go:117] "RemoveContainer" containerID="7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec"
Sep 09 12:31:20.197030 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:31:20.197012    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec\": container with ID starting with 7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec not found: ID does not exist" containerID="7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec"
Sep 09 12:31:20.197118 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.197037    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec"} err="failed to get container status \"7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec\": rpc error: code = NotFound desc = could not find container \"7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec\": container with ID starting with 7da858175628066b80a1d93422bc0cd4d180832f4e6a2447328853a60867dfec not found: ID does not exist"
Sep 09 12:31:20.223080 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.223054    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"]
Sep 09 12:31:20.236374 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:20.236303    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-db67bc-openshift-default-588df66f5b-vp6vv"]
Sep 09 12:31:21.901043 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:21.901006    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cd8f6a5-238b-4be8-837d-caf16221558d" path="/var/lib/kubelet/pods/9cd8f6a5-238b-4be8-837d-caf16221558d/volumes"
Sep 09 12:31:38.886341 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.886301    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"]
Sep 09 12:31:38.887008 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.886988    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd8f6a5-238b-4be8-837d-caf16221558d" containerName="istio-proxy"
Sep 09 12:31:38.887096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.887010    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd8f6a5-238b-4be8-837d-caf16221558d" containerName="istio-proxy"
Sep 09 12:31:38.887096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.887040    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="364eb190-c9e4-4f34-8b8f-efd15708e5f6" containerName="main"
Sep 09 12:31:38.887096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.887049    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="364eb190-c9e4-4f34-8b8f-efd15708e5f6" containerName="main"
Sep 09 12:31:38.887242 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.887185    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="364eb190-c9e4-4f34-8b8f-efd15708e5f6" containerName="main"
Sep 09 12:31:38.887242 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.887200    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd8f6a5-238b-4be8-837d-caf16221558d" containerName="istio-proxy"
Sep 09 12:31:38.890319 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.890296    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:38.892842 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.892819    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-bc0270\"/\"distinct-2-w4-bc0270-kserve-self-signed-certs\""
Sep 09 12:31:38.899245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.899216    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"]
Sep 09 12:31:38.990536 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.990496    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-home\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:38.990747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.990589    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-model-cache\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:38.990747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.990616    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-dshm\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:38.990747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.990634    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-tmp-dir\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:38.990876 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.990750    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxrcp\" (UniqueName: \"kubernetes.io/projected/bcc4611a-3bdf-4961-864f-76b26d745c64-kube-api-access-kxrcp\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:38.990876 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:38.990768    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcc4611a-3bdf-4961-864f-76b26d745c64-tls-certs\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.091886 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.091844    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-dshm\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.092076 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.091899    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-tmp-dir\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.092076 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.092012    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxrcp\" (UniqueName: \"kubernetes.io/projected/bcc4611a-3bdf-4961-864f-76b26d745c64-kube-api-access-kxrcp\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.092076 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.092044    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcc4611a-3bdf-4961-864f-76b26d745c64-tls-certs\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.092243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.092090    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-home\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.092243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.092198    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-model-cache\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.092431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.092409    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-tmp-dir\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.092536 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.092516    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-model-cache\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.092632 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.092611    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-home\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.094639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.094601    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-dshm\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.094833 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.094815    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcc4611a-3bdf-4961-864f-76b26d745c64-tls-certs\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.102080 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.102056    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxrcp\" (UniqueName: \"kubernetes.io/projected/bcc4611a-3bdf-4961-864f-76b26d745c64-kube-api-access-kxrcp\") pod \"distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") " pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.202018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.201921    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:39.477542 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:39.477510    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"]
Sep 09 12:31:39.478958 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:31:39.478923    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcc4611a_3bdf_4961_864f_76b26d745c64.slice/crio-e5d67b13ff3cc011949453a722cd2c7f14fb33618bc825fe7ad82b2c30f47d51 WatchSource:0}: Error finding container e5d67b13ff3cc011949453a722cd2c7f14fb33618bc825fe7ad82b2c30f47d51: Status 404 returned error can't find the container with id e5d67b13ff3cc011949453a722cd2c7f14fb33618bc825fe7ad82b2c30f47d51
Sep 09 12:31:40.312986 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:40.312953    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj" event={"ID":"bcc4611a-3bdf-4961-864f-76b26d745c64","Type":"ContainerStarted","Data":"72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422"}
Sep 09 12:31:40.313387 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:40.313001    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj" event={"ID":"bcc4611a-3bdf-4961-864f-76b26d745c64","Type":"ContainerStarted","Data":"e5d67b13ff3cc011949453a722cd2c7f14fb33618bc825fe7ad82b2c30f47d51"}
Sep 09 12:31:40.337768 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:40.337670    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj" podStartSLOduration=2.337647844 podStartE2EDuration="2.337647844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:31:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:31:40.337135912 +0000 UTC m=+1244.978755130" watchObservedRunningTime="2026-09-09 12:31:40.337647844 +0000 UTC m=+1244.979267049"
Sep 09 12:31:41.696084 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:41.696043    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"]
Sep 09 12:31:41.699858 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:41.699837    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"
Sep 09 12:31:41.702054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:41.702034    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:31:41.702149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:41.702037    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:31:41.708718 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:41.708666    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"]
Sep 09 12:31:41.824191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:41.824150    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1298d01b-ab65-4184-9c9e-ae31ad4ff7d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx\" (UID: \"1298d01b-ab65-4184-9c9e-ae31ad4ff7d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"
Sep 09 12:31:41.925108 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:41.925067    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1298d01b-ab65-4184-9c9e-ae31ad4ff7d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx\" (UID: \"1298d01b-ab65-4184-9c9e-ae31ad4ff7d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"
Sep 09 12:31:41.925687 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:41.925665    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1298d01b-ab65-4184-9c9e-ae31ad4ff7d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx\" (UID: \"1298d01b-ab65-4184-9c9e-ae31ad4ff7d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"
Sep 09 12:31:42.012883 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:42.012841    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"
Sep 09 12:31:42.297717 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:42.297678    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"]
Sep 09 12:31:42.299504 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:31:42.299471    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1298d01b_ab65_4184_9c9e_ae31ad4ff7d7.slice/crio-7b7ea00790a79e4e6d70b7b32035f2f614a87d00004c751430601af4034c9992 WatchSource:0}: Error finding container 7b7ea00790a79e4e6d70b7b32035f2f614a87d00004c751430601af4034c9992: Status 404 returned error can't find the container with id 7b7ea00790a79e4e6d70b7b32035f2f614a87d00004c751430601af4034c9992
Sep 09 12:31:42.326136 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:42.326106    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx" event={"ID":"1298d01b-ab65-4184-9c9e-ae31ad4ff7d7","Type":"ContainerStarted","Data":"7b7ea00790a79e4e6d70b7b32035f2f614a87d00004c751430601af4034c9992"}
Sep 09 12:31:42.415555 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:42.415521    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"]
Sep 09 12:31:43.333552 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:43.333459    2585 generic.go:356] "Generic (PLEG): container finished" podID="1298d01b-ab65-4184-9c9e-ae31ad4ff7d7" containerID="9f932a6e1bf09701bcf56d1bbc6056f4a9f6ea68547ef475fc8f62373ee01b98" exitCode=0
Sep 09 12:31:43.333931 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:43.333560    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx" event={"ID":"1298d01b-ab65-4184-9c9e-ae31ad4ff7d7","Type":"ContainerDied","Data":"9f932a6e1bf09701bcf56d1bbc6056f4a9f6ea68547ef475fc8f62373ee01b98"}
Sep 09 12:31:44.480610 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:44.480582    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"
Sep 09 12:31:44.656965 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:44.656871    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1298d01b-ab65-4184-9c9e-ae31ad4ff7d7-maas-api-service-ca\") pod \"1298d01b-ab65-4184-9c9e-ae31ad4ff7d7\" (UID: \"1298d01b-ab65-4184-9c9e-ae31ad4ff7d7\") "
Sep 09 12:31:44.657308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:44.657287    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1298d01b-ab65-4184-9c9e-ae31ad4ff7d7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1298d01b-ab65-4184-9c9e-ae31ad4ff7d7" (UID: "1298d01b-ab65-4184-9c9e-ae31ad4ff7d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:31:44.758645 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:44.758607    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1298d01b-ab65-4184-9c9e-ae31ad4ff7d7-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:45.347627 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:45.347597    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"
Sep 09 12:31:45.347852 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:45.347634    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx" event={"ID":"1298d01b-ab65-4184-9c9e-ae31ad4ff7d7","Type":"ContainerDied","Data":"7b7ea00790a79e4e6d70b7b32035f2f614a87d00004c751430601af4034c9992"}
Sep 09 12:31:45.347852 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:45.347683    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b7ea00790a79e4e6d70b7b32035f2f614a87d00004c751430601af4034c9992"
Sep 09 12:31:49.202779 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:49.202740    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:49.203167 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:49.202793    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:49.215036 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:49.215007    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:49.383419 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:49.383387    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:31:52.551950 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.551911    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"]
Sep 09 12:31:52.552469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.552130    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f" podUID="a229715f-540f-4e60-9af2-8cb6ed1a0c37" containerName="payload-processing" containerID="cri-o://2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a" gracePeriod=30
Sep 09 12:31:52.741958 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.741923    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"]
Sep 09 12:31:52.748482 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.748446    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-2a142a-4wmsx"]
Sep 09 12:31:52.814039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.813966    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:31:52.838462 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.838426    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5gpp\" (UniqueName: \"kubernetes.io/projected/a229715f-540f-4e60-9af2-8cb6ed1a0c37-kube-api-access-h5gpp\") pod \"a229715f-540f-4e60-9af2-8cb6ed1a0c37\" (UID: \"a229715f-540f-4e60-9af2-8cb6ed1a0c37\") "
Sep 09 12:31:52.838761 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.838525    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a229715f-540f-4e60-9af2-8cb6ed1a0c37-plugins-config-volume\") pod \"a229715f-540f-4e60-9af2-8cb6ed1a0c37\" (UID: \"a229715f-540f-4e60-9af2-8cb6ed1a0c37\") "
Sep 09 12:31:52.838925 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.838901    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a229715f-540f-4e60-9af2-8cb6ed1a0c37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a229715f-540f-4e60-9af2-8cb6ed1a0c37" (UID: "a229715f-540f-4e60-9af2-8cb6ed1a0c37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:31:52.839018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.839000    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a229715f-540f-4e60-9af2-8cb6ed1a0c37-plugins-config-volume\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:52.840836 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.840801    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a229715f-540f-4e60-9af2-8cb6ed1a0c37-kube-api-access-h5gpp" (OuterVolumeSpecName: "kube-api-access-h5gpp") pod "a229715f-540f-4e60-9af2-8cb6ed1a0c37" (UID: "a229715f-540f-4e60-9af2-8cb6ed1a0c37"). InnerVolumeSpecName "kube-api-access-h5gpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:31:52.940234 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:52.940186    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5gpp\" (UniqueName: \"kubernetes.io/projected/a229715f-540f-4e60-9af2-8cb6ed1a0c37-kube-api-access-h5gpp\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:53.398330 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.398298    2585 generic.go:356] "Generic (PLEG): container finished" podID="a229715f-540f-4e60-9af2-8cb6ed1a0c37" containerID="2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a" exitCode=0
Sep 09 12:31:53.398518 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.398346    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"
Sep 09 12:31:53.398518 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.398353    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f" event={"ID":"a229715f-540f-4e60-9af2-8cb6ed1a0c37","Type":"ContainerDied","Data":"2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a"}
Sep 09 12:31:53.398518 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.398394    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f" event={"ID":"a229715f-540f-4e60-9af2-8cb6ed1a0c37","Type":"ContainerDied","Data":"c0e6045eda63467ac697577b358a4b43a51e47872f5dab4b433de8a197342f78"}
Sep 09 12:31:53.398518 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.398414    2585 scope.go:117] "RemoveContainer" containerID="2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a"
Sep 09 12:31:53.408020 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.407999    2585 scope.go:117] "RemoveContainer" containerID="2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a"
Sep 09 12:31:53.408301 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:31:53.408282    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a\": container with ID starting with 2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a not found: ID does not exist" containerID="2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a"
Sep 09 12:31:53.408357 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.408310    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a"} err="failed to get container status \"2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a\": rpc error: code = NotFound desc = could not find container \"2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a\": container with ID starting with 2d67bb43cf5dbc0838e0e4caaa617ccd5c932673a25a0761048befa471c4667a not found: ID does not exist"
Sep 09 12:31:53.422521 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.422488    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"]
Sep 09 12:31:53.426949 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.426925    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2a142a-59fdd6c69c-2lp2f"]
Sep 09 12:31:53.900167 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.900137    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1298d01b-ab65-4184-9c9e-ae31ad4ff7d7" path="/var/lib/kubelet/pods/1298d01b-ab65-4184-9c9e-ae31ad4ff7d7/volumes"
Sep 09 12:31:53.900547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:53.900529    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a229715f-540f-4e60-9af2-8cb6ed1a0c37" path="/var/lib/kubelet/pods/a229715f-540f-4e60-9af2-8cb6ed1a0c37/volumes"
Sep 09 12:31:55.968678 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:55.968647    2585 scope.go:117] "RemoveContainer" containerID="86d1381a8d23ee347859c689b0121d0f287059f0d11d67950520ac4c0f19de9e"
Sep 09 12:31:55.977606 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:55.977585    2585 scope.go:117] "RemoveContainer" containerID="aa79d6dc1cea207e5a201eab333696cc6ee7aa03dff2965c92a26136a879084e"
Sep 09 12:31:57.436515 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.436476    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz" podUID="774222e7-be56-43ab-8877-9c3e8c4e21d7" containerName="main" containerID="cri-o://b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741" gracePeriod=45
Sep 09 12:31:57.689355 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.689296    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:31:57.794731 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.794678    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-tmp-dir\") pod \"774222e7-be56-43ab-8877-9c3e8c4e21d7\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") "
Sep 09 12:31:57.794923 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.794744    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xbsr\" (UniqueName: \"kubernetes.io/projected/774222e7-be56-43ab-8877-9c3e8c4e21d7-kube-api-access-8xbsr\") pod \"774222e7-be56-43ab-8877-9c3e8c4e21d7\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") "
Sep 09 12:31:57.794923 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.794824    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-home\") pod \"774222e7-be56-43ab-8877-9c3e8c4e21d7\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") "
Sep 09 12:31:57.794923 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.794906    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-dshm\") pod \"774222e7-be56-43ab-8877-9c3e8c4e21d7\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") "
Sep 09 12:31:57.795084 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.794960    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/774222e7-be56-43ab-8877-9c3e8c4e21d7-tls-certs\") pod \"774222e7-be56-43ab-8877-9c3e8c4e21d7\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") "
Sep 09 12:31:57.795084 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.794963    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "774222e7-be56-43ab-8877-9c3e8c4e21d7" (UID: "774222e7-be56-43ab-8877-9c3e8c4e21d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:57.795084 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.795065    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-model-cache\") pod \"774222e7-be56-43ab-8877-9c3e8c4e21d7\" (UID: \"774222e7-be56-43ab-8877-9c3e8c4e21d7\") "
Sep 09 12:31:57.795242 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.795088    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-home" (OuterVolumeSpecName: "home") pod "774222e7-be56-43ab-8877-9c3e8c4e21d7" (UID: "774222e7-be56-43ab-8877-9c3e8c4e21d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:57.795328 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.795301    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-model-cache" (OuterVolumeSpecName: "model-cache") pod "774222e7-be56-43ab-8877-9c3e8c4e21d7" (UID: "774222e7-be56-43ab-8877-9c3e8c4e21d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:57.795483 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.795461    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:57.795483 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.795484    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:57.795667 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.795497    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:57.797354 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.797312    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/774222e7-be56-43ab-8877-9c3e8c4e21d7-kube-api-access-8xbsr" (OuterVolumeSpecName: "kube-api-access-8xbsr") pod "774222e7-be56-43ab-8877-9c3e8c4e21d7" (UID: "774222e7-be56-43ab-8877-9c3e8c4e21d7"). InnerVolumeSpecName "kube-api-access-8xbsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:31:57.797454 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.797361    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/774222e7-be56-43ab-8877-9c3e8c4e21d7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "774222e7-be56-43ab-8877-9c3e8c4e21d7" (UID: "774222e7-be56-43ab-8877-9c3e8c4e21d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:31:57.797454 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.797367    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-dshm" (OuterVolumeSpecName: "dshm") pod "774222e7-be56-43ab-8877-9c3e8c4e21d7" (UID: "774222e7-be56-43ab-8877-9c3e8c4e21d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:31:57.896537 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.896502    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/774222e7-be56-43ab-8877-9c3e8c4e21d7-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:57.896537 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.896529    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/774222e7-be56-43ab-8877-9c3e8c4e21d7-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:57.896537 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:57.896539    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xbsr\" (UniqueName: \"kubernetes.io/projected/774222e7-be56-43ab-8877-9c3e8c4e21d7-kube-api-access-8xbsr\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:31:58.429760 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:58.429726    2585 generic.go:356] "Generic (PLEG): container finished" podID="774222e7-be56-43ab-8877-9c3e8c4e21d7" containerID="b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741" exitCode=0
Sep 09 12:31:58.429760 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:58.429755    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"
Sep 09 12:31:58.429975 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:58.429789    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz" event={"ID":"774222e7-be56-43ab-8877-9c3e8c4e21d7","Type":"ContainerDied","Data":"b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741"}
Sep 09 12:31:58.429975 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:58.429834    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz" event={"ID":"774222e7-be56-43ab-8877-9c3e8c4e21d7","Type":"ContainerDied","Data":"914cbf3ab355e02e1197fc53a3b19b30f8accf01a6a3ad758fe651e747487a05"}
Sep 09 12:31:58.429975 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:58.429856    2585 scope.go:117] "RemoveContainer" containerID="b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741"
Sep 09 12:31:58.439282 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:58.439106    2585 scope.go:117] "RemoveContainer" containerID="b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741"
Sep 09 12:31:58.439513 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:31:58.439394    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741\": container with ID starting with b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741 not found: ID does not exist" containerID="b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741"
Sep 09 12:31:58.439513 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:58.439418    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741"} err="failed to get container status \"b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741\": rpc error: code = NotFound desc = could not find container \"b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741\": container with ID starting with b2506b1225979f7d6cb67712d61c446a6984b49481eb87ee2f3df80bd88ea741 not found: ID does not exist"
Sep 09 12:31:58.448739 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:58.448674    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"]
Sep 09 12:31:58.454599 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:58.454567    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2a142a/test-model-2a142a-kserve-cb94775ff-whgbz"]
Sep 09 12:31:59.899917 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:31:59.899864    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="774222e7-be56-43ab-8877-9c3e8c4e21d7" path="/var/lib/kubelet/pods/774222e7-be56-43ab-8877-9c3e8c4e21d7/volumes"
Sep 09 12:32:00.426394 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:00.426363    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"]
Sep 09 12:32:00.426594 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:00.426574    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy" containerID="cri-o://930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5" gracePeriod=30
Sep 09 12:32:01.330836 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.330798    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"]
Sep 09 12:32:01.331438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.331414    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="774222e7-be56-43ab-8877-9c3e8c4e21d7" containerName="main"
Sep 09 12:32:01.331438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.331435    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="774222e7-be56-43ab-8877-9c3e8c4e21d7" containerName="main"
Sep 09 12:32:01.331584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.331447    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1298d01b-ab65-4184-9c9e-ae31ad4ff7d7" containerName="revoke-keys"
Sep 09 12:32:01.331584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.331453    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1298d01b-ab65-4184-9c9e-ae31ad4ff7d7" containerName="revoke-keys"
Sep 09 12:32:01.331584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.331461    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a229715f-540f-4e60-9af2-8cb6ed1a0c37" containerName="payload-processing"
Sep 09 12:32:01.331584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.331466    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a229715f-540f-4e60-9af2-8cb6ed1a0c37" containerName="payload-processing"
Sep 09 12:32:01.331584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.331532    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="774222e7-be56-43ab-8877-9c3e8c4e21d7" containerName="main"
Sep 09 12:32:01.331584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.331539    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a229715f-540f-4e60-9af2-8cb6ed1a0c37" containerName="payload-processing"
Sep 09 12:32:01.331584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.331545    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1298d01b-ab65-4184-9c9e-ae31ad4ff7d7" containerName="revoke-keys"
Sep 09 12:32:01.336052 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.336030    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.338616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.338599    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-679f45-openshift-default-dockercfg-crptj\""
Sep 09 12:32:01.354911 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.354873    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"]
Sep 09 12:32:01.435036 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.434977    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-envoy\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.435036 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.435028    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7k5n\" (UniqueName: \"kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-kube-api-access-p7k5n\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.435425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.435061    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-credential-socket\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.435425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.435086    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-istio-token\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.435425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.435116    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-socket\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.435425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.435191    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-data\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.435425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.435231    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-certs\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.435425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.435257    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b517406b-c239-4ff5-837f-354d938c575b-istiod-ca-cert\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.435425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.435381    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b517406b-c239-4ff5-837f-354d938c575b-istio-podinfo\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.536608 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.536564    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b517406b-c239-4ff5-837f-354d938c575b-istio-podinfo\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.536914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.536624    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-envoy\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.536914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.536651    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7k5n\" (UniqueName: \"kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-kube-api-access-p7k5n\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.536914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.536682    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-credential-socket\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.536914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.536741    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-istio-token\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.536914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.536774    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-socket\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.536914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.536845    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-data\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.536914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.536885    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-certs\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.536914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.536910    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b517406b-c239-4ff5-837f-354d938c575b-istiod-ca-cert\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.537356 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.537176    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-credential-socket\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.537356 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.537206    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-socket\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.537635 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.537378    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-data\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.537635 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.537421    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-certs\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.537635 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.537623    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b517406b-c239-4ff5-837f-354d938c575b-istiod-ca-cert\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.539168 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.539147    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-envoy\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.539395 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.539376    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b517406b-c239-4ff5-837f-354d938c575b-istio-podinfo\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.548185 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.548152    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-istio-token\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.548336 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.548270    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7k5n\" (UniqueName: \"kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-kube-api-access-p7k5n\") pod \"e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") " pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.646937 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.646835    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:01.934602 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.934569    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"]
Sep 09 12:32:01.936162 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:32:01.936131    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb517406b_c239_4ff5_837f_354d938c575b.slice/crio-68085e13f2d12732825e0195a5034cee213ce0d9ea68525b29b7a3004516f7bb WatchSource:0}: Error finding container 68085e13f2d12732825e0195a5034cee213ce0d9ea68525b29b7a3004516f7bb: Status 404 returned error can't find the container with id 68085e13f2d12732825e0195a5034cee213ce0d9ea68525b29b7a3004516f7bb
Sep 09 12:32:01.938620 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.938589    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:32:01.938725 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.938655    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:32:01.938725 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:01.938683    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:32:02.459748 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:02.459710    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" event={"ID":"b517406b-c239-4ff5-837f-354d938c575b","Type":"ContainerStarted","Data":"cfc50fc3f5c118da1e9588ca60b59257625af2999381b423a1eb556964f3c3ca"}
Sep 09 12:32:02.460244 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:02.459777    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" event={"ID":"b517406b-c239-4ff5-837f-354d938c575b","Type":"ContainerStarted","Data":"68085e13f2d12732825e0195a5034cee213ce0d9ea68525b29b7a3004516f7bb"}
Sep 09 12:32:02.491634 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:02.491581    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" podStartSLOduration=1.491558688 podStartE2EDuration="1.491558688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:32:02.49010057 +0000 UTC m=+1267.131719775" watchObservedRunningTime="2026-09-09 12:32:02.491558688 +0000 UTC m=+1267.133177893"
Sep 09 12:32:02.647008 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:02.646963    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:02.716848 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:02.716763    2585 patch_prober.go:28] interesting pod/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 12:32:02.716848 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:02.716828    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:32:03.652870 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:03.652830    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:04.472147 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:04.472114    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:04.473012 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:04.472993    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:05.671840 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.671809    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:32:05.789526 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.789496    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-token\") pod \"d480be69-2c10-4eeb-9819-56ac690fb8c4\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") "
Sep 09 12:32:05.789761 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.789580    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-socket\") pod \"d480be69-2c10-4eeb-9819-56ac690fb8c4\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") "
Sep 09 12:32:05.789761 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.789624    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-envoy\") pod \"d480be69-2c10-4eeb-9819-56ac690fb8c4\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") "
Sep 09 12:32:05.789761 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.789657    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-podinfo\") pod \"d480be69-2c10-4eeb-9819-56ac690fb8c4\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") "
Sep 09 12:32:05.789761 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.789689    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcpcm\" (UniqueName: \"kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-kube-api-access-rcpcm\") pod \"d480be69-2c10-4eeb-9819-56ac690fb8c4\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") "
Sep 09 12:32:05.789761 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.789749    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-credential-socket\") pod \"d480be69-2c10-4eeb-9819-56ac690fb8c4\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") "
Sep 09 12:32:05.790151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.789780    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-data\") pod \"d480be69-2c10-4eeb-9819-56ac690fb8c4\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") "
Sep 09 12:32:05.790151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.789832    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-certs\") pod \"d480be69-2c10-4eeb-9819-56ac690fb8c4\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") "
Sep 09 12:32:05.790151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.789911    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d480be69-2c10-4eeb-9819-56ac690fb8c4-istiod-ca-cert\") pod \"d480be69-2c10-4eeb-9819-56ac690fb8c4\" (UID: \"d480be69-2c10-4eeb-9819-56ac690fb8c4\") "
Sep 09 12:32:05.790151 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.789916    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d480be69-2c10-4eeb-9819-56ac690fb8c4" (UID: "d480be69-2c10-4eeb-9819-56ac690fb8c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:32:05.790385 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.790360    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:32:05.790765 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.790739    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "d480be69-2c10-4eeb-9819-56ac690fb8c4" (UID: "d480be69-2c10-4eeb-9819-56ac690fb8c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:32:05.791164 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.791110    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d480be69-2c10-4eeb-9819-56ac690fb8c4" (UID: "d480be69-2c10-4eeb-9819-56ac690fb8c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:32:05.791639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.791603    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d480be69-2c10-4eeb-9819-56ac690fb8c4" (UID: "d480be69-2c10-4eeb-9819-56ac690fb8c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:32:05.791819 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.791799    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d480be69-2c10-4eeb-9819-56ac690fb8c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d480be69-2c10-4eeb-9819-56ac690fb8c4" (UID: "d480be69-2c10-4eeb-9819-56ac690fb8c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:32:05.792802 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.792765    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-kube-api-access-rcpcm" (OuterVolumeSpecName: "kube-api-access-rcpcm") pod "d480be69-2c10-4eeb-9819-56ac690fb8c4" (UID: "d480be69-2c10-4eeb-9819-56ac690fb8c4"). InnerVolumeSpecName "kube-api-access-rcpcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:32:05.793411 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.793386    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d480be69-2c10-4eeb-9819-56ac690fb8c4" (UID: "d480be69-2c10-4eeb-9819-56ac690fb8c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:32:05.798112 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.798087    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "d480be69-2c10-4eeb-9819-56ac690fb8c4" (UID: "d480be69-2c10-4eeb-9819-56ac690fb8c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:32:05.798261 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.798243    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d480be69-2c10-4eeb-9819-56ac690fb8c4" (UID: "d480be69-2c10-4eeb-9819-56ac690fb8c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:32:05.891655 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.891617    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d480be69-2c10-4eeb-9819-56ac690fb8c4-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:32:05.891655 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.891648    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:32:05.891655 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.891657    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:32:05.891655 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.891665    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:32:05.891967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.891673    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcpcm\" (UniqueName: \"kubernetes.io/projected/d480be69-2c10-4eeb-9819-56ac690fb8c4-kube-api-access-rcpcm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:32:05.891967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.891682    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:32:05.891967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.891690    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:32:05.891967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:05.891725    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d480be69-2c10-4eeb-9819-56ac690fb8c4-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:32:06.487250 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.487215    2585 generic.go:356] "Generic (PLEG): container finished" podID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerID="930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5" exitCode=0
Sep 09 12:32:06.487469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.487272    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"
Sep 09 12:32:06.487469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.487317    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" event={"ID":"d480be69-2c10-4eeb-9819-56ac690fb8c4","Type":"ContainerDied","Data":"930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5"}
Sep 09 12:32:06.487469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.487388    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb" event={"ID":"d480be69-2c10-4eeb-9819-56ac690fb8c4","Type":"ContainerDied","Data":"a7084b75fceac73756d46f64f05d6d38be2ba9552d44bf48172bf7bdb431cd07"}
Sep 09 12:32:06.487469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.487419    2585 scope.go:117] "RemoveContainer" containerID="930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5"
Sep 09 12:32:06.497146 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.497128    2585 scope.go:117] "RemoveContainer" containerID="06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1"
Sep 09 12:32:06.505739 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.505717    2585 scope.go:117] "RemoveContainer" containerID="930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5"
Sep 09 12:32:06.506005 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:32:06.505990    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5\": container with ID starting with 930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5 not found: ID does not exist" containerID="930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5"
Sep 09 12:32:06.506064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.506014    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5"} err="failed to get container status \"930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5\": rpc error: code = NotFound desc = could not find container \"930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5\": container with ID starting with 930141d37685cc851acd2a7818360d1ed6f6a36504f9a65550394714ed0131e5 not found: ID does not exist"
Sep 09 12:32:06.506064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.506028    2585 scope.go:117] "RemoveContainer" containerID="06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1"
Sep 09 12:32:06.506303 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:32:06.506287    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1\": container with ID starting with 06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1 not found: ID does not exist" containerID="06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1"
Sep 09 12:32:06.506350 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.506305    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1"} err="failed to get container status \"06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1\": rpc error: code = NotFound desc = could not find container \"06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1\": container with ID starting with 06e2a9da64242d75237ae062605cf858f48b07fd5992d7e84f91c666de8422e1 not found: ID does not exist"
Sep 09 12:32:06.514891 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.514867    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"]
Sep 09 12:32:06.520213 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.520191    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-2a142a-openshift-default-6f97f8bb86-jnqqb"]
Sep 09 12:32:06.679522 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.679490    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"]
Sep 09 12:32:06.679972 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.679958    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy"
Sep 09 12:32:06.679972 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.679974    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy"
Sep 09 12:32:06.680063 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.679983    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy"
Sep 09 12:32:06.680063 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.679988    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy"
Sep 09 12:32:06.680134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.680064    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy"
Sep 09 12:32:06.680134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.680074    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" containerName="istio-proxy"
Sep 09 12:32:06.684665 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.684646    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:32:06.688082 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.688063    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-679f45\""
Sep 09 12:32:06.688270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.688251    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-679f45-dockercfg-9w585\""
Sep 09 12:32:06.701189 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.701164    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"]
Sep 09 12:32:06.801679 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.801642    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm\" (UID: \"a3ad359b-cd7a-46db-aca2-3de2aa4e189e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:32:06.801903 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.801739    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsrsw\" (UniqueName: \"kubernetes.io/projected/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-kube-api-access-xsrsw\") pod \"payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm\" (UID: \"a3ad359b-cd7a-46db-aca2-3de2aa4e189e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:32:06.902948 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.902914    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm\" (UID: \"a3ad359b-cd7a-46db-aca2-3de2aa4e189e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:32:06.903157 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.902971    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsrsw\" (UniqueName: \"kubernetes.io/projected/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-kube-api-access-xsrsw\") pod \"payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm\" (UID: \"a3ad359b-cd7a-46db-aca2-3de2aa4e189e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:32:06.903646 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.903621    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm\" (UID: \"a3ad359b-cd7a-46db-aca2-3de2aa4e189e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:32:06.912797 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.912773    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsrsw\" (UniqueName: \"kubernetes.io/projected/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-kube-api-access-xsrsw\") pod \"payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm\" (UID: \"a3ad359b-cd7a-46db-aca2-3de2aa4e189e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:32:06.995539 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:06.995505    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:32:07.280887 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:07.280861    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"]
Sep 09 12:32:07.283132 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:32:07.283107    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3ad359b_cd7a_46db_aca2_3de2aa4e189e.slice/crio-a876b38de14e36c951765f89572a2436957dd5616e5264f3f6a75f1592e95c57 WatchSource:0}: Error finding container a876b38de14e36c951765f89572a2436957dd5616e5264f3f6a75f1592e95c57: Status 404 returned error can't find the container with id a876b38de14e36c951765f89572a2436957dd5616e5264f3f6a75f1592e95c57
Sep 09 12:32:07.496893 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:07.496849    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm" event={"ID":"a3ad359b-cd7a-46db-aca2-3de2aa4e189e","Type":"ContainerStarted","Data":"be92b27d393e6f06f01c305fd4d2b23c39df02f21c7c7870424a0f9ce757d075"}
Sep 09 12:32:07.497065 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:07.496911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm" event={"ID":"a3ad359b-cd7a-46db-aca2-3de2aa4e189e","Type":"ContainerStarted","Data":"a876b38de14e36c951765f89572a2436957dd5616e5264f3f6a75f1592e95c57"}
Sep 09 12:32:07.497065 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:07.497019    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:32:07.519977 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:07.519934    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm" podStartSLOduration=1.519918337 podStartE2EDuration="1.519918337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:32:07.517034331 +0000 UTC m=+1272.158653558" watchObservedRunningTime="2026-09-09 12:32:07.519918337 +0000 UTC m=+1272.161537541"
Sep 09 12:32:07.900104 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:07.900074    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d480be69-2c10-4eeb-9819-56ac690fb8c4" path="/var/lib/kubelet/pods/d480be69-2c10-4eeb-9819-56ac690fb8c4/volumes"
Sep 09 12:32:18.504316 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:18.504284    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:32:30.280771 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.276915    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"]
Sep 09 12:32:30.281563 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.281534    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.286618 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.286404    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-cfcdfd-openshift-default-dockercfg-qrjh7\""
Sep 09 12:32:30.301398 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.301369    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"]
Sep 09 12:32:30.348067 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.348025    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-envoy\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.348252 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.348093    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-socket\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.348252 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.348144    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d50c73-3bdd-4c5e-8948-81daf607b301-istiod-ca-cert\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.348252 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.348186    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-data\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.348252 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.348239    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-certs\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.348489 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.348259    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-podinfo\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.348489 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.348327    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-token\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.348489 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.348384    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-credential-socket\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.348489 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.348406    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxjmb\" (UniqueName: \"kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-kube-api-access-rxjmb\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.449663 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.449615    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-envoy\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.449912 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.449689    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-socket\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.449912 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.449754    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d50c73-3bdd-4c5e-8948-81daf607b301-istiod-ca-cert\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.449912 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.449780    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-data\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.449912 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.449830    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-certs\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.449912 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.449855    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-podinfo\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.449912 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.449911    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-token\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.450266 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.449972    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-credential-socket\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.450266 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.450006    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxjmb\" (UniqueName: \"kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-kube-api-access-rxjmb\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.450266 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.450123    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-socket\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.450428 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.450346    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-certs\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.451034 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.451006    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d50c73-3bdd-4c5e-8948-81daf607b301-istiod-ca-cert\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.451182 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.451156    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-data\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.451510 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.451486    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-credential-socket\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.456247 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.456217    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-podinfo\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.456519 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.456496    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-envoy\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.465194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.465164    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-token\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.468316 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.468286    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxjmb\" (UniqueName: \"kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-kube-api-access-rxjmb\") pod \"e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") " pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.595504 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.595412    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:30.894606 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.894581    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"]
Sep 09 12:32:30.896604 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:32:30.896576    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d50c73_3bdd_4c5e_8948_81daf607b301.slice/crio-575c48354caf0c358589022f7cf241d7040da2fda5b2ed7357a80d45a8b84bdb WatchSource:0}: Error finding container 575c48354caf0c358589022f7cf241d7040da2fda5b2ed7357a80d45a8b84bdb: Status 404 returned error can't find the container with id 575c48354caf0c358589022f7cf241d7040da2fda5b2ed7357a80d45a8b84bdb
Sep 09 12:32:30.898960 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.898926    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:32:30.899105 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.899070    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:32:30.899221 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:30.899134    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:32:31.680772 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:31.680734    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" event={"ID":"a1d50c73-3bdd-4c5e-8948-81daf607b301","Type":"ContainerStarted","Data":"7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5"}
Sep 09 12:32:31.681161 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:31.680804    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" event={"ID":"a1d50c73-3bdd-4c5e-8948-81daf607b301","Type":"ContainerStarted","Data":"575c48354caf0c358589022f7cf241d7040da2fda5b2ed7357a80d45a8b84bdb"}
Sep 09 12:32:31.726153 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:31.726089    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" podStartSLOduration=1.7260692450000001 podStartE2EDuration="1.726069245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:32:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:32:31.723890636 +0000 UTC m=+1296.365509842" watchObservedRunningTime="2026-09-09 12:32:31.726069245 +0000 UTC m=+1296.367688450"
Sep 09 12:32:32.596111 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:32.596018    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:32.602468 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:32.602439    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:32.688207 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:32.688175    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:32.689341 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:32.689320    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:32:35.159249 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:35.158241    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": read tcp 10.132.0.2:51792->10.132.0.64:15021: read: connection reset by peer" start-of-body=
Sep 09 12:32:35.159249 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:35.158307    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": read tcp 10.132.0.2:51792->10.132.0.64:15021: read: connection reset by peer"
Sep 09 12:32:35.710356 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:35.710326    2585 generic.go:356] "Generic (PLEG): container finished" podID="b517406b-c239-4ff5-837f-354d938c575b" containerID="cfc50fc3f5c118da1e9588ca60b59257625af2999381b423a1eb556964f3c3ca" exitCode=0
Sep 09 12:32:35.710557 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:35.710402    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" event={"ID":"b517406b-c239-4ff5-837f-354d938c575b","Type":"ContainerDied","Data":"cfc50fc3f5c118da1e9588ca60b59257625af2999381b423a1eb556964f3c3ca"}
Sep 09 12:32:35.710861 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:35.710843    2585 scope.go:117] "RemoveContainer" containerID="cfc50fc3f5c118da1e9588ca60b59257625af2999381b423a1eb556964f3c3ca"
Sep 09 12:32:35.712046 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:35.712008    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:32:35.712138 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:35.712083    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:32:35.712138 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:35.712113    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:32:36.717943 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:36.717911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" event={"ID":"b517406b-c239-4ff5-837f-354d938c575b","Type":"ContainerStarted","Data":"c2f32df505e0375c97e118f0d9374fcf26438fcb3ee9a8853639abbd0b42afbf"}
Sep 09 12:32:37.647818 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:37.647787    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:37.647989 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:37.647953    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 09 12:32:37.648037 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:37.648002    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 09 12:32:38.647930 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:38.647893    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 09 12:32:38.648398 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:38.647969    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 09 12:32:39.648121 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:39.648081    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 09 12:32:39.648603 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:39.648147    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 09 12:32:40.647429 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:40.647390    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 09 12:32:40.647612 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:40.647452    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 09 12:32:41.647463 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:41.647423    2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 09 12:32:41.647874 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:41.647489    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 09 12:32:42.651971 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:42.651933    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:42.652437 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:42.652271    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:42.653073 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:42.653056    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:32:56.017527 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:32:56.017488    2585 scope.go:117] "RemoveContainer" containerID="ca121154d2f7f245b0a0b6375f731ec69a6593e9456abc591674f616ddf0d9af"
Sep 09 12:33:00.867638 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:00.867603    2585 generic.go:356] "Generic (PLEG): container finished" podID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerID="7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5" exitCode=0
Sep 09 12:33:00.868087 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:00.867668    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" event={"ID":"a1d50c73-3bdd-4c5e-8948-81daf607b301","Type":"ContainerDied","Data":"7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5"}
Sep 09 12:33:00.868134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:00.868119    2585 scope.go:117] "RemoveContainer" containerID="7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5"
Sep 09 12:33:00.869389 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:00.869359    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:33:00.869504 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:00.869437    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:33:00.869504 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:00.869482    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:33:01.875868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:01.875829    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" event={"ID":"a1d50c73-3bdd-4c5e-8948-81daf607b301","Type":"ContainerStarted","Data":"629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638"}
Sep 09 12:33:02.596443 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:02.596398    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:33:02.597738 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:02.597684    2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:33:02.597849 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:02.597771    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 09 12:33:02.689073 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:02.689034    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:33:03.596518 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:03.596484    2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:33:03.596957 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:03.596538    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 09 12:33:04.596205 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:04.596166    2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:33:04.596407 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:04.596226    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 09 12:33:05.596198 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:05.596160    2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:33:05.596611 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:05.596225    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 09 12:33:06.596211 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:06.596170    2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:33:06.596613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:06.596241    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 09 12:33:07.596540 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:07.596504    2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:33:07.597028 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:07.596566    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 09 12:33:08.600722 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:08.600677    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:33:08.601805 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:08.601787    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:33:26.805837 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.805763    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"]
Sep 09 12:33:26.809473 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.809457    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.812482 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.812456    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-c554db\"/\"unconfigured-w0-c554db-kserve-self-signed-certs\""
Sep 09 12:33:26.822433 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.822411    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"]
Sep 09 12:33:26.863374 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.863346    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e099851-c987-435f-84d5-4519576c4614-tls-certs\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.863553 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.863406    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-tmp-dir\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.863553 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.863492    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-dshm\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.863553 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.863529    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-home\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.863692 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.863562    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-model-cache\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.863692 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.863659    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lml2\" (UniqueName: \"kubernetes.io/projected/4e099851-c987-435f-84d5-4519576c4614-kube-api-access-9lml2\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.965180 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.965138    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-tmp-dir\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.965390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.965217    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-dshm\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.965390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.965246    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-home\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.965390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.965275    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-model-cache\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.965390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.965332    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lml2\" (UniqueName: \"kubernetes.io/projected/4e099851-c987-435f-84d5-4519576c4614-kube-api-access-9lml2\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.965606 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.965431    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e099851-c987-435f-84d5-4519576c4614-tls-certs\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.965661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.965615    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-tmp-dir\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.965951 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.965923    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-model-cache\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.966754 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.966728    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-home\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.969464 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.969352    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-dshm\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.970061 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.970037    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e099851-c987-435f-84d5-4519576c4614-tls-certs\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:26.977425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:26.977402    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lml2\" (UniqueName: \"kubernetes.io/projected/4e099851-c987-435f-84d5-4519576c4614-kube-api-access-9lml2\") pod \"unconfigured-w0-c554db-kserve-75b844f4f5-krh52\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") " pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:27.121610 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:27.121525    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:27.397430 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:27.397397    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"]
Sep 09 12:33:27.399321 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:33:27.399296    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e099851_c987_435f_84d5_4519576c4614.slice/crio-d3c233e6eee513a0dbabc95f3dcc836a3efd9dd6ba9acb7d4f947a04b4eac1dc WatchSource:0}: Error finding container d3c233e6eee513a0dbabc95f3dcc836a3efd9dd6ba9acb7d4f947a04b4eac1dc: Status 404 returned error can't find the container with id d3c233e6eee513a0dbabc95f3dcc836a3efd9dd6ba9acb7d4f947a04b4eac1dc
Sep 09 12:33:28.036125 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:28.036085    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52" event={"ID":"4e099851-c987-435f-84d5-4519576c4614","Type":"ContainerStarted","Data":"140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035"}
Sep 09 12:33:28.036125 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:28.036128    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52" event={"ID":"4e099851-c987-435f-84d5-4519576c4614","Type":"ContainerStarted","Data":"d3c233e6eee513a0dbabc95f3dcc836a3efd9dd6ba9acb7d4f947a04b4eac1dc"}
Sep 09 12:33:28.057443 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:28.057384    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52" podStartSLOduration=2.057370389 podStartE2EDuration="2.057370389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:28.055493367 +0000 UTC m=+1352.697112574" watchObservedRunningTime="2026-09-09 12:33:28.057370389 +0000 UTC m=+1352.698989593"
Sep 09 12:33:36.908839 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:36.908794    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"]
Sep 09 12:33:36.912922 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:36.912896    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:36.915376 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:36.915355    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-bc0270\"/\"unconfigured-w4-bc0270-kserve-self-signed-certs\""
Sep 09 12:33:36.924139 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:36.924112    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"]
Sep 09 12:33:36.979098 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:36.979068    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-dshm\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:36.979269 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:36.979179    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-tmp-dir\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:36.979269 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:36.979232    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-model-cache\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:36.979369 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:36.979288    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-home\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:36.979369 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:36.979305    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0de13caa-9563-4faa-bd6f-6784747f7813-tls-certs\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:36.979452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:36.979402    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8gxk\" (UniqueName: \"kubernetes.io/projected/0de13caa-9563-4faa-bd6f-6784747f7813-kube-api-access-f8gxk\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.080198 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.080160    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-tmp-dir\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.080198 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.080206    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-model-cache\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.080482 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.080250    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-home\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.080482 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.080265    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0de13caa-9563-4faa-bd6f-6784747f7813-tls-certs\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.080482 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.080316    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8gxk\" (UniqueName: \"kubernetes.io/projected/0de13caa-9563-4faa-bd6f-6784747f7813-kube-api-access-f8gxk\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.080482 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.080362    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-dshm\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.081147 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.081058    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-home\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.081358 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.081334    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-tmp-dir\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.081455 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.081438    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-model-cache\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.083386 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.083326    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-dshm\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.085666 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.085618    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0de13caa-9563-4faa-bd6f-6784747f7813-tls-certs\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.094060 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.094031    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8gxk\" (UniqueName: \"kubernetes.io/projected/0de13caa-9563-4faa-bd6f-6784747f7813-kube-api-access-f8gxk\") pod \"unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") " pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.121899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.121856    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:37.122069 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.121923    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:37.134859 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.134830    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:37.223860 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.223778    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:37.509354 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:37.509323    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"]
Sep 09 12:33:37.511272 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:33:37.511240    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0de13caa_9563_4faa_bd6f_6784747f7813.slice/crio-15b0884b14b27755c450d43b5b77f131e110c0330b5cf4e3b2a0dcd7767b99b7 WatchSource:0}: Error finding container 15b0884b14b27755c450d43b5b77f131e110c0330b5cf4e3b2a0dcd7767b99b7: Status 404 returned error can't find the container with id 15b0884b14b27755c450d43b5b77f131e110c0330b5cf4e3b2a0dcd7767b99b7
Sep 09 12:33:38.102424 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:38.102372    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx" event={"ID":"0de13caa-9563-4faa-bd6f-6784747f7813","Type":"ContainerStarted","Data":"c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409"}
Sep 09 12:33:38.102944 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:38.102435    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx" event={"ID":"0de13caa-9563-4faa-bd6f-6784747f7813","Type":"ContainerStarted","Data":"15b0884b14b27755c450d43b5b77f131e110c0330b5cf4e3b2a0dcd7767b99b7"}
Sep 09 12:33:38.114458 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:38.114432    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:33:38.122050 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:38.122008    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx" podStartSLOduration=2.121994101 podStartE2EDuration="2.121994101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:38.120751144 +0000 UTC m=+1362.762370346" watchObservedRunningTime="2026-09-09 12:33:38.121994101 +0000 UTC m=+1362.763613304"
Sep 09 12:33:47.223957 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:47.223916    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:47.224439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:47.223983    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:47.236588 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:47.236554    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:48.171864 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:48.171831    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:33:56.050682 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:56.050639    2585 scope.go:117] "RemoveContainer" containerID="4b53275ec37b9d23142c4c2503ebe6ea2a5b3681d3fce3fa26d203da57b98467"
Sep 09 12:33:56.059730 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:33:56.059686    2585 scope.go:117] "RemoveContainer" containerID="f6994e4f4190b2315437ac0ebadd9671e1379d5dae771093873444f6e6160812"
Sep 09 12:34:27.489396 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:27.489350    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"]
Sep 09 12:34:27.494975 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:27.494947    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"
Sep 09 12:34:27.498719 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:27.498496    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:34:27.498719 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:27.498634    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:34:27.501284 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:27.501259    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"]
Sep 09 12:34:27.514033 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:27.514006    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f31d601-a6ce-4d90-bcbe-3698ec963ed3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv\" (UID: \"5f31d601-a6ce-4d90-bcbe-3698ec963ed3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"
Sep 09 12:34:27.615001 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:27.614967    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f31d601-a6ce-4d90-bcbe-3698ec963ed3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv\" (UID: \"5f31d601-a6ce-4d90-bcbe-3698ec963ed3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"
Sep 09 12:34:27.615584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:27.615563    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f31d601-a6ce-4d90-bcbe-3698ec963ed3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv\" (UID: \"5f31d601-a6ce-4d90-bcbe-3698ec963ed3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"
Sep 09 12:34:27.809772 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:27.809731    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"
Sep 09 12:34:28.089669 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:28.089642    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"]
Sep 09 12:34:28.091666 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:34:28.091636    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f31d601_a6ce_4d90_bcbe_3698ec963ed3.slice/crio-f855489073fbe1e3d3686dd2ca0ab37fade8096e6be53d9be3bf4f276f0023b4 WatchSource:0}: Error finding container f855489073fbe1e3d3686dd2ca0ab37fade8096e6be53d9be3bf4f276f0023b4: Status 404 returned error can't find the container with id f855489073fbe1e3d3686dd2ca0ab37fade8096e6be53d9be3bf4f276f0023b4
Sep 09 12:34:28.395675 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:28.395641    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv" event={"ID":"5f31d601-a6ce-4d90-bcbe-3698ec963ed3","Type":"ContainerStarted","Data":"9cb1468fa0f3300833d8da16f2e5ab3443cb8f7d39a6a4f129e4d0a55f3fdc27"}
Sep 09 12:34:28.395869 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:28.395720    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv" event={"ID":"5f31d601-a6ce-4d90-bcbe-3698ec963ed3","Type":"ContainerStarted","Data":"f855489073fbe1e3d3686dd2ca0ab37fade8096e6be53d9be3bf4f276f0023b4"}
Sep 09 12:34:28.413933 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:28.413884    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv" podStartSLOduration=1.413866961 podStartE2EDuration="1.413866961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:34:28.413024396 +0000 UTC m=+1413.054643603" watchObservedRunningTime="2026-09-09 12:34:28.413866961 +0000 UTC m=+1413.055486164"
Sep 09 12:34:29.403492 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:29.403459    2585 generic.go:356] "Generic (PLEG): container finished" podID="5f31d601-a6ce-4d90-bcbe-3698ec963ed3" containerID="9cb1468fa0f3300833d8da16f2e5ab3443cb8f7d39a6a4f129e4d0a55f3fdc27" exitCode=0
Sep 09 12:34:29.403891 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:29.403537    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv" event={"ID":"5f31d601-a6ce-4d90-bcbe-3698ec963ed3","Type":"ContainerDied","Data":"9cb1468fa0f3300833d8da16f2e5ab3443cb8f7d39a6a4f129e4d0a55f3fdc27"}
Sep 09 12:34:30.548727 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:30.548670    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"
Sep 09 12:34:30.645790 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:30.645754    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f31d601-a6ce-4d90-bcbe-3698ec963ed3-maas-api-service-ca\") pod \"5f31d601-a6ce-4d90-bcbe-3698ec963ed3\" (UID: \"5f31d601-a6ce-4d90-bcbe-3698ec963ed3\") "
Sep 09 12:34:30.646173 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:30.646149    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f31d601-a6ce-4d90-bcbe-3698ec963ed3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f31d601-a6ce-4d90-bcbe-3698ec963ed3" (UID: "5f31d601-a6ce-4d90-bcbe-3698ec963ed3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:34:30.747790 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:30.747663    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f31d601-a6ce-4d90-bcbe-3698ec963ed3-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:34:31.416901 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:31.416866    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv" event={"ID":"5f31d601-a6ce-4d90-bcbe-3698ec963ed3","Type":"ContainerDied","Data":"f855489073fbe1e3d3686dd2ca0ab37fade8096e6be53d9be3bf4f276f0023b4"}
Sep 09 12:34:31.417083 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:31.416909    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"
Sep 09 12:34:31.417083 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:31.416927    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f855489073fbe1e3d3686dd2ca0ab37fade8096e6be53d9be3bf4f276f0023b4"
Sep 09 12:34:43.327064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:43.327024    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"]
Sep 09 12:34:43.330958 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:43.330929    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24ec44a5-x4vrv"]
Sep 09 12:34:43.900293 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:34:43.900261    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f31d601-a6ce-4d90-bcbe-3698ec963ed3" path="/var/lib/kubelet/pods/5f31d601-a6ce-4d90-bcbe-3698ec963ed3/volumes"
Sep 09 12:35:01.991475 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:01.991438    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"]
Sep 09 12:35:01.997087 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:01.997048    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"]
Sep 09 12:35:02.006677 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:02.006650    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"]
Sep 09 12:35:02.021239 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:02.021193    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"]
Sep 09 12:35:13.338214 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.338168    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"]
Sep 09 12:35:13.338735 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.338402    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl" podUID="3453ea77-0da0-4e6a-a413-8ddb50448697" containerName="payload-pre-processing" containerID="cri-o://4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea" gracePeriod=30
Sep 09 12:35:13.604411 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.604348    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:35:13.690056 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.690021    2585 generic.go:356] "Generic (PLEG): container finished" podID="3453ea77-0da0-4e6a-a413-8ddb50448697" containerID="4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea" exitCode=0
Sep 09 12:35:13.690230 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.690071    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl" event={"ID":"3453ea77-0da0-4e6a-a413-8ddb50448697","Type":"ContainerDied","Data":"4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea"}
Sep 09 12:35:13.690230 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.690117    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl" event={"ID":"3453ea77-0da0-4e6a-a413-8ddb50448697","Type":"ContainerDied","Data":"531400f5da1a989143efcf7934c3b87b49e75caa0acaacfe4b151958a127e634"}
Sep 09 12:35:13.690230 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.690140    2585 scope.go:117] "RemoveContainer" containerID="4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea"
Sep 09 12:35:13.690357 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.690276    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"
Sep 09 12:35:13.702981 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.702962    2585 scope.go:117] "RemoveContainer" containerID="4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea"
Sep 09 12:35:13.703549 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:35:13.703481    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea\": container with ID starting with 4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea not found: ID does not exist" containerID="4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea"
Sep 09 12:35:13.703751 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.703520    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea"} err="failed to get container status \"4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea\": rpc error: code = NotFound desc = could not find container \"4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea\": container with ID starting with 4294edf884bde655bad5a3c6aa8bce8b0e6f943483d420eafa85415623853aea not found: ID does not exist"
Sep 09 12:35:13.720075 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.720046    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvftd\" (UniqueName: \"kubernetes.io/projected/3453ea77-0da0-4e6a-a413-8ddb50448697-kube-api-access-zvftd\") pod \"3453ea77-0da0-4e6a-a413-8ddb50448697\" (UID: \"3453ea77-0da0-4e6a-a413-8ddb50448697\") "
Sep 09 12:35:13.720243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.720232    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3453ea77-0da0-4e6a-a413-8ddb50448697-plugins-config-volume\") pod \"3453ea77-0da0-4e6a-a413-8ddb50448697\" (UID: \"3453ea77-0da0-4e6a-a413-8ddb50448697\") "
Sep 09 12:35:13.720577 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.720548    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3453ea77-0da0-4e6a-a413-8ddb50448697-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3453ea77-0da0-4e6a-a413-8ddb50448697" (UID: "3453ea77-0da0-4e6a-a413-8ddb50448697"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:35:13.722353 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.722322    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3453ea77-0da0-4e6a-a413-8ddb50448697-kube-api-access-zvftd" (OuterVolumeSpecName: "kube-api-access-zvftd") pod "3453ea77-0da0-4e6a-a413-8ddb50448697" (UID: "3453ea77-0da0-4e6a-a413-8ddb50448697"). InnerVolumeSpecName "kube-api-access-zvftd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:13.822079 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.822045    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3453ea77-0da0-4e6a-a413-8ddb50448697-plugins-config-volume\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:13.822079 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:13.822073    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvftd\" (UniqueName: \"kubernetes.io/projected/3453ea77-0da0-4e6a-a413-8ddb50448697-kube-api-access-zvftd\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:14.008792 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:14.008760    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"]
Sep 09 12:35:14.013839 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:14.013813    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c554db-dc79f7886-zp7bl"]
Sep 09 12:35:15.900065 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:15.900024    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3453ea77-0da0-4e6a-a413-8ddb50448697" path="/var/lib/kubelet/pods/3453ea77-0da0-4e6a-a413-8ddb50448697/volumes"
Sep 09 12:35:17.020831 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.020794    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5" podUID="c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" containerName="main" containerID="cri-o://37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15" gracePeriod=45
Sep 09 12:35:17.029163 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.029130    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f" podUID="66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" containerName="main" containerID="cri-o://feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7" gracePeriod=45
Sep 09 12:35:17.042839 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.042808    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52" podUID="4e099851-c987-435f-84d5-4519576c4614" containerName="main" containerID="cri-o://140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035" gracePeriod=45
Sep 09 12:35:17.305741 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.305692    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:35:17.341663 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.341324    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:35:17.353126 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.353104    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:35:17.461007 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.460973    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-model-cache\") pod \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") "
Sep 09 12:35:17.461194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461023    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-dshm\") pod \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") "
Sep 09 12:35:17.461194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461090    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-model-cache\") pod \"4e099851-c987-435f-84d5-4519576c4614\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") "
Sep 09 12:35:17.461194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461133    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lml2\" (UniqueName: \"kubernetes.io/projected/4e099851-c987-435f-84d5-4519576c4614-kube-api-access-9lml2\") pod \"4e099851-c987-435f-84d5-4519576c4614\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") "
Sep 09 12:35:17.461194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461181    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tmp-dir\") pod \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") "
Sep 09 12:35:17.461417 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461213    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-home\") pod \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") "
Sep 09 12:35:17.461417 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461239    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-tmp-dir\") pod \"4e099851-c987-435f-84d5-4519576c4614\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") "
Sep 09 12:35:17.461417 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461263    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-model-cache" (OuterVolumeSpecName: "model-cache") pod "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" (UID: "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.461417 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461307    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tls-certs\") pod \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") "
Sep 09 12:35:17.461417 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461346    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e099851-c987-435f-84d5-4519576c4614" (UID: "4e099851-c987-435f-84d5-4519576c4614"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.461417 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461390    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e099851-c987-435f-84d5-4519576c4614-tls-certs\") pod \"4e099851-c987-435f-84d5-4519576c4614\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") "
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461423    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-dshm\") pod \"4e099851-c987-435f-84d5-4519576c4614\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") "
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461462    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tmp-dir\") pod \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") "
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461498    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-home" (OuterVolumeSpecName: "home") pod "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" (UID: "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461503    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-home\") pod \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") "
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461547    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrqbp\" (UniqueName: \"kubernetes.io/projected/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-kube-api-access-xrqbp\") pod \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") "
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461591    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tls-certs\") pod \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") "
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461635    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-home" (OuterVolumeSpecName: "home") pod "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" (UID: "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461647    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p29sx\" (UniqueName: \"kubernetes.io/projected/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-kube-api-access-p29sx\") pod \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\" (UID: \"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6\") "
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461673    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-dshm\") pod \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") "
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461748    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-model-cache\") pod \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\" (UID: \"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6\") "
Sep 09 12:35:17.461810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.461789    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-home\") pod \"4e099851-c987-435f-84d5-4519576c4614\" (UID: \"4e099851-c987-435f-84d5-4519576c4614\") "
Sep 09 12:35:17.462524 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.462054    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e099851-c987-435f-84d5-4519576c4614" (UID: "4e099851-c987-435f-84d5-4519576c4614"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.462524 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.462278    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" (UID: "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.462524 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.462447    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.462524 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.462465    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.462524 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.462479    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.462524 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.462491    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.462524 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.462503    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.462524 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.462514    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.463250 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.462823    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" (UID: "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.463250 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.462963    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-model-cache" (OuterVolumeSpecName: "model-cache") pod "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" (UID: "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.463443 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.463366    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-home" (OuterVolumeSpecName: "home") pod "4e099851-c987-435f-84d5-4519576c4614" (UID: "4e099851-c987-435f-84d5-4519576c4614"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.464416 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.464390    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-dshm" (OuterVolumeSpecName: "dshm") pod "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" (UID: "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.464766 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.464726    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" (UID: "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:35:17.465218 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.465194    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-kube-api-access-xrqbp" (OuterVolumeSpecName: "kube-api-access-xrqbp") pod "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" (UID: "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6"). InnerVolumeSpecName "kube-api-access-xrqbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:17.466129 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.465862    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-dshm" (OuterVolumeSpecName: "dshm") pod "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" (UID: "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.466129 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.466096    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" (UID: "c90e5774-cb91-45a7-ae31-3a1a8c2a48e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:35:17.466129 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.466114    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-dshm" (OuterVolumeSpecName: "dshm") pod "4e099851-c987-435f-84d5-4519576c4614" (UID: "4e099851-c987-435f-84d5-4519576c4614"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.466436 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.466410    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e099851-c987-435f-84d5-4519576c4614-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e099851-c987-435f-84d5-4519576c4614" (UID: "4e099851-c987-435f-84d5-4519576c4614"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:35:17.467178 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.467153    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e099851-c987-435f-84d5-4519576c4614-kube-api-access-9lml2" (OuterVolumeSpecName: "kube-api-access-9lml2") pod "4e099851-c987-435f-84d5-4519576c4614" (UID: "4e099851-c987-435f-84d5-4519576c4614"). InnerVolumeSpecName "kube-api-access-9lml2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:17.467481 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.467459    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-kube-api-access-p29sx" (OuterVolumeSpecName: "kube-api-access-p29sx") pod "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" (UID: "66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6"). InnerVolumeSpecName "kube-api-access-p29sx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:17.563263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563181    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563210    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e099851-c987-435f-84d5-4519576c4614-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563218    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563226    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563235    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrqbp\" (UniqueName: \"kubernetes.io/projected/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-kube-api-access-xrqbp\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563244    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563252    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p29sx\" (UniqueName: \"kubernetes.io/projected/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-kube-api-access-p29sx\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563260    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563268    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563664 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563276    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e099851-c987-435f-84d5-4519576c4614-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563664 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563283    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.563664 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.563290    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lml2\" (UniqueName: \"kubernetes.io/projected/4e099851-c987-435f-84d5-4519576c4614-kube-api-access-9lml2\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.633448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.633398    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67" podUID="615e2500-2db0-4d7c-9a69-69348178f6a6" containerName="main" containerID="cri-o://1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9" gracePeriod=45
Sep 09 12:35:17.715984 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.715955    2585 generic.go:356] "Generic (PLEG): container finished" podID="c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" containerID="37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15" exitCode=0
Sep 09 12:35:17.716142 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.716004    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"
Sep 09 12:35:17.716142 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.716035    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5" event={"ID":"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6","Type":"ContainerDied","Data":"37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15"}
Sep 09 12:35:17.716142 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.716080    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5" event={"ID":"c90e5774-cb91-45a7-ae31-3a1a8c2a48e6","Type":"ContainerDied","Data":"c837f85b0f4f4ac3cced713b09f6925494e9f03b6b9103baafa91a9dcc73c585"}
Sep 09 12:35:17.716142 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.716097    2585 scope.go:117] "RemoveContainer" containerID="37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15"
Sep 09 12:35:17.717582 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.717561    2585 generic.go:356] "Generic (PLEG): container finished" podID="66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" containerID="feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7" exitCode=0
Sep 09 12:35:17.717670 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.717609    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"
Sep 09 12:35:17.717670 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.717639    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f" event={"ID":"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6","Type":"ContainerDied","Data":"feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7"}
Sep 09 12:35:17.717815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.717684    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f" event={"ID":"66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6","Type":"ContainerDied","Data":"6d832a3fadfe0153754f54e5540c023fbf983391af2b41ec6c0c5748b926e789"}
Sep 09 12:35:17.718980 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.718958    2585 generic.go:356] "Generic (PLEG): container finished" podID="4e099851-c987-435f-84d5-4519576c4614" containerID="140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035" exitCode=0
Sep 09 12:35:17.719083 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.719005    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"
Sep 09 12:35:17.719083 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.719054    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52" event={"ID":"4e099851-c987-435f-84d5-4519576c4614","Type":"ContainerDied","Data":"140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035"}
Sep 09 12:35:17.719160 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.719097    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52" event={"ID":"4e099851-c987-435f-84d5-4519576c4614","Type":"ContainerDied","Data":"d3c233e6eee513a0dbabc95f3dcc836a3efd9dd6ba9acb7d4f947a04b4eac1dc"}
Sep 09 12:35:17.744305 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.744280    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"]
Sep 09 12:35:17.745434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.745416    2585 scope.go:117] "RemoveContainer" containerID="37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15"
Sep 09 12:35:17.745722 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:35:17.745685    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15\": container with ID starting with 37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15 not found: ID does not exist" containerID="37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15"
Sep 09 12:35:17.745810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.745736    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15"} err="failed to get container status \"37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15\": rpc error: code = NotFound desc = could not find container \"37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15\": container with ID starting with 37a192803bca6a8a121d2aac3742c1e521451db381908ddc486611fb28fd0f15 not found: ID does not exist"
Sep 09 12:35:17.745810 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.745755    2585 scope.go:117] "RemoveContainer" containerID="feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7"
Sep 09 12:35:17.750504 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.750474    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-w0-c554db-kserve-56bfb5fcdd-7d6z5"]
Sep 09 12:35:17.756222 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.756198    2585 scope.go:117] "RemoveContainer" containerID="feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7"
Sep 09 12:35:17.756527 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:35:17.756508    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7\": container with ID starting with feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7 not found: ID does not exist" containerID="feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7"
Sep 09 12:35:17.756594 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.756535    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7"} err="failed to get container status \"feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7\": rpc error: code = NotFound desc = could not find container \"feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7\": container with ID starting with feae8d8f21d3c33f1f930e478c8077c44a5a74fa56a5f29e649eabd943b11fa7 not found: ID does not exist"
Sep 09 12:35:17.756594 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.756552    2585 scope.go:117] "RemoveContainer" containerID="140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035"
Sep 09 12:35:17.763041 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.763019    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"]
Sep 09 12:35:17.766852 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.766827    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/premium-w0-c554db-kserve-6cbdc7bd5-fqs8f"]
Sep 09 12:35:17.767850 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.767833    2585 scope.go:117] "RemoveContainer" containerID="140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035"
Sep 09 12:35:17.768161 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:35:17.768144    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035\": container with ID starting with 140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035 not found: ID does not exist" containerID="140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035"
Sep 09 12:35:17.768228 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.768165    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035"} err="failed to get container status \"140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035\": rpc error: code = NotFound desc = could not find container \"140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035\": container with ID starting with 140d37cbe1f4dde2687e659eaecef125637ebc939d884c19f57f0e4188c7c035 not found: ID does not exist"
Sep 09 12:35:17.778020 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.777992    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"]
Sep 09 12:35:17.782768 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.782742    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/unconfigured-w0-c554db-kserve-75b844f4f5-krh52"]
Sep 09 12:35:17.867689 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.867665    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:35:17.901424 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.901392    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e099851-c987-435f-84d5-4519576c4614" path="/var/lib/kubelet/pods/4e099851-c987-435f-84d5-4519576c4614/volumes"
Sep 09 12:35:17.902007 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.901987    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" path="/var/lib/kubelet/pods/66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6/volumes"
Sep 09 12:35:17.902534 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.902516    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" path="/var/lib/kubelet/pods/c90e5774-cb91-45a7-ae31-3a1a8c2a48e6/volumes"
Sep 09 12:35:17.967143 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967114    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-model-cache\") pod \"615e2500-2db0-4d7c-9a69-69348178f6a6\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") "
Sep 09 12:35:17.967307 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967168    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5vff\" (UniqueName: \"kubernetes.io/projected/615e2500-2db0-4d7c-9a69-69348178f6a6-kube-api-access-w5vff\") pod \"615e2500-2db0-4d7c-9a69-69348178f6a6\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") "
Sep 09 12:35:17.967307 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967193    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-dshm\") pod \"615e2500-2db0-4d7c-9a69-69348178f6a6\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") "
Sep 09 12:35:17.967307 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967238    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/615e2500-2db0-4d7c-9a69-69348178f6a6-tls-certs\") pod \"615e2500-2db0-4d7c-9a69-69348178f6a6\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") "
Sep 09 12:35:17.967307 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967295    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-home\") pod \"615e2500-2db0-4d7c-9a69-69348178f6a6\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") "
Sep 09 12:35:17.967531 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967323    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-tmp-dir\") pod \"615e2500-2db0-4d7c-9a69-69348178f6a6\" (UID: \"615e2500-2db0-4d7c-9a69-69348178f6a6\") "
Sep 09 12:35:17.967531 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967350    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "615e2500-2db0-4d7c-9a69-69348178f6a6" (UID: "615e2500-2db0-4d7c-9a69-69348178f6a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.967745 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967545    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-home" (OuterVolumeSpecName: "home") pod "615e2500-2db0-4d7c-9a69-69348178f6a6" (UID: "615e2500-2db0-4d7c-9a69-69348178f6a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.967814 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967738    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "615e2500-2db0-4d7c-9a69-69348178f6a6" (UID: "615e2500-2db0-4d7c-9a69-69348178f6a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.968038 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967888    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.968038 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967910    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.968038 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.967924    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:17.969414 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.969397    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-dshm" (OuterVolumeSpecName: "dshm") pod "615e2500-2db0-4d7c-9a69-69348178f6a6" (UID: "615e2500-2db0-4d7c-9a69-69348178f6a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:17.969897 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.969878    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/615e2500-2db0-4d7c-9a69-69348178f6a6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "615e2500-2db0-4d7c-9a69-69348178f6a6" (UID: "615e2500-2db0-4d7c-9a69-69348178f6a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:35:17.969897 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:17.969884    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615e2500-2db0-4d7c-9a69-69348178f6a6-kube-api-access-w5vff" (OuterVolumeSpecName: "kube-api-access-w5vff") pod "615e2500-2db0-4d7c-9a69-69348178f6a6" (UID: "615e2500-2db0-4d7c-9a69-69348178f6a6"). InnerVolumeSpecName "kube-api-access-w5vff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:18.069302 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.069267    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5vff\" (UniqueName: \"kubernetes.io/projected/615e2500-2db0-4d7c-9a69-69348178f6a6-kube-api-access-w5vff\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:18.069302 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.069293    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/615e2500-2db0-4d7c-9a69-69348178f6a6-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:18.069302 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.069304    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/615e2500-2db0-4d7c-9a69-69348178f6a6-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:18.728490 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.728458    2585 generic.go:356] "Generic (PLEG): container finished" podID="615e2500-2db0-4d7c-9a69-69348178f6a6" containerID="1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9" exitCode=0
Sep 09 12:35:18.728652 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.728505    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"
Sep 09 12:35:18.728652 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.728547    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67" event={"ID":"615e2500-2db0-4d7c-9a69-69348178f6a6","Type":"ContainerDied","Data":"1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9"}
Sep 09 12:35:18.728652 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.728593    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67" event={"ID":"615e2500-2db0-4d7c-9a69-69348178f6a6","Type":"ContainerDied","Data":"589ec58661d80988f39eb8d1db08e836d8e74dd8b1503c041f5df13231cf91ea"}
Sep 09 12:35:18.728652 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.728613    2585 scope.go:117] "RemoveContainer" containerID="1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9"
Sep 09 12:35:18.737683 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.737667    2585 scope.go:117] "RemoveContainer" containerID="1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9"
Sep 09 12:35:18.737956 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:35:18.737938    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9\": container with ID starting with 1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9 not found: ID does not exist" containerID="1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9"
Sep 09 12:35:18.738033 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.737962    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9"} err="failed to get container status \"1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9\": rpc error: code = NotFound desc = could not find container \"1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9\": container with ID starting with 1f616670d2059952deff23823b9248676db7f5acdbeee0b5423f5ec82555c7a9 not found: ID does not exist"
Sep 09 12:35:18.751960 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.751931    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"]
Sep 09 12:35:18.757354 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:18.757331    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-c554db/distinct-2-w0-c554db-kserve-f78699c9f-p4b67"]
Sep 09 12:35:19.900803 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:19.900768    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="615e2500-2db0-4d7c-9a69-69348178f6a6" path="/var/lib/kubelet/pods/615e2500-2db0-4d7c-9a69-69348178f6a6/volumes"
Sep 09 12:35:20.857220 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:20.857185    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"]
Sep 09 12:35:20.860467 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:20.859127    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy" containerID="cri-o://4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c" gracePeriod=30
Sep 09 12:35:26.126412 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.126383    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:35:26.255539 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255510    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-envoy\") pod \"76595636-4e31-433e-86de-3c4df5e8ed8a\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") "
Sep 09 12:35:26.255747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255563    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-socket\") pod \"76595636-4e31-433e-86de-3c4df5e8ed8a\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") "
Sep 09 12:35:26.255747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255590    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-certs\") pod \"76595636-4e31-433e-86de-3c4df5e8ed8a\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") "
Sep 09 12:35:26.255747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255672    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76595636-4e31-433e-86de-3c4df5e8ed8a-istiod-ca-cert\") pod \"76595636-4e31-433e-86de-3c4df5e8ed8a\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") "
Sep 09 12:35:26.255747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255716    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-podinfo\") pod \"76595636-4e31-433e-86de-3c4df5e8ed8a\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") "
Sep 09 12:35:26.255971 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255757    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-data\") pod \"76595636-4e31-433e-86de-3c4df5e8ed8a\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") "
Sep 09 12:35:26.255971 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255793    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-token\") pod \"76595636-4e31-433e-86de-3c4df5e8ed8a\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") "
Sep 09 12:35:26.255971 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255869    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-credential-socket\") pod \"76595636-4e31-433e-86de-3c4df5e8ed8a\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") "
Sep 09 12:35:26.255971 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255908    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5hjt\" (UniqueName: \"kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-kube-api-access-d5hjt\") pod \"76595636-4e31-433e-86de-3c4df5e8ed8a\" (UID: \"76595636-4e31-433e-86de-3c4df5e8ed8a\") "
Sep 09 12:35:26.255971 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255925    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76595636-4e31-433e-86de-3c4df5e8ed8a" (UID: "76595636-4e31-433e-86de-3c4df5e8ed8a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:26.256204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.255958    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76595636-4e31-433e-86de-3c4df5e8ed8a" (UID: "76595636-4e31-433e-86de-3c4df5e8ed8a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:26.256204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.256066    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-data" (OuterVolumeSpecName: "istio-data") pod "76595636-4e31-433e-86de-3c4df5e8ed8a" (UID: "76595636-4e31-433e-86de-3c4df5e8ed8a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:26.256204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.256139    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76595636-4e31-433e-86de-3c4df5e8ed8a" (UID: "76595636-4e31-433e-86de-3c4df5e8ed8a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:26.256204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.256183    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76595636-4e31-433e-86de-3c4df5e8ed8a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76595636-4e31-433e-86de-3c4df5e8ed8a" (UID: "76595636-4e31-433e-86de-3c4df5e8ed8a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:35:26.256417 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.256404    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:26.256469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.256421    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:26.256469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.256435    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:26.256469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.256448    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76595636-4e31-433e-86de-3c4df5e8ed8a-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:26.256469 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.256459    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:26.258247 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.258218    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76595636-4e31-433e-86de-3c4df5e8ed8a" (UID: "76595636-4e31-433e-86de-3c4df5e8ed8a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:26.258365 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.258320    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76595636-4e31-433e-86de-3c4df5e8ed8a" (UID: "76595636-4e31-433e-86de-3c4df5e8ed8a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:35:26.258738 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.258715    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-token" (OuterVolumeSpecName: "istio-token") pod "76595636-4e31-433e-86de-3c4df5e8ed8a" (UID: "76595636-4e31-433e-86de-3c4df5e8ed8a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:26.258852 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.258837    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-kube-api-access-d5hjt" (OuterVolumeSpecName: "kube-api-access-d5hjt") pod "76595636-4e31-433e-86de-3c4df5e8ed8a" (UID: "76595636-4e31-433e-86de-3c4df5e8ed8a"). InnerVolumeSpecName "kube-api-access-d5hjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:26.357438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.357407    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:26.357438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.357433    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:26.357438 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.357443    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5hjt\" (UniqueName: \"kubernetes.io/projected/76595636-4e31-433e-86de-3c4df5e8ed8a-kube-api-access-d5hjt\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:26.357658 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.357453    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76595636-4e31-433e-86de-3c4df5e8ed8a-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:35:26.779141 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.779113    2585 generic.go:356] "Generic (PLEG): container finished" podID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerID="4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c" exitCode=0
Sep 09 12:35:26.779283 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.779155    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" event={"ID":"76595636-4e31-433e-86de-3c4df5e8ed8a","Type":"ContainerDied","Data":"4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c"}
Sep 09 12:35:26.779283 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.779193    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt" event={"ID":"76595636-4e31-433e-86de-3c4df5e8ed8a","Type":"ContainerDied","Data":"cacb641841cb053627706628a4b8947b65fb0e11e14dd0ce80625842b4d7aec7"}
Sep 09 12:35:26.779283 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.779215    2585 scope.go:117] "RemoveContainer" containerID="4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c"
Sep 09 12:35:26.779283 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.779159    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"
Sep 09 12:35:26.792157 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.792136    2585 scope.go:117] "RemoveContainer" containerID="abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e"
Sep 09 12:35:26.800853 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.800836    2585 scope.go:117] "RemoveContainer" containerID="4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c"
Sep 09 12:35:26.801108 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:35:26.801092    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c\": container with ID starting with 4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c not found: ID does not exist" containerID="4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c"
Sep 09 12:35:26.801149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.801117    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c"} err="failed to get container status \"4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c\": rpc error: code = NotFound desc = could not find container \"4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c\": container with ID starting with 4ee0ea23b24fe33549fc6464b339e32cdde745ddb8258d100a4d7381f616f56c not found: ID does not exist"
Sep 09 12:35:26.801149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.801131    2585 scope.go:117] "RemoveContainer" containerID="abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e"
Sep 09 12:35:26.801405 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:35:26.801388    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e\": container with ID starting with abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e not found: ID does not exist" containerID="abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e"
Sep 09 12:35:26.801439 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.801412    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e"} err="failed to get container status \"abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e\": rpc error: code = NotFound desc = could not find container \"abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e\": container with ID starting with abbf25d6a59e3dc31097b5c3eda51a672fdc610ced0d2980a0eae8ab0aa2b84e not found: ID does not exist"
Sep 09 12:35:26.805086 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.805064    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"]
Sep 09 12:35:26.809555 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:26.809536    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-c554db-openshift-default-7b4856b694-gwtgt"]
Sep 09 12:35:27.899965 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:27.899930    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" path="/var/lib/kubelet/pods/76595636-4e31-433e-86de-3c4df5e8ed8a/volumes"
Sep 09 12:35:37.199263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.199226    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"]
Sep 09 12:35:37.199944 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.199912    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3453ea77-0da0-4e6a-a413-8ddb50448697" containerName="payload-pre-processing"
Sep 09 12:35:37.199944 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.199941    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3453ea77-0da0-4e6a-a413-8ddb50448697" containerName="payload-pre-processing"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.199956    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f31d601-a6ce-4d90-bcbe-3698ec963ed3" containerName="revoke-keys"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.199964    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f31d601-a6ce-4d90-bcbe-3698ec963ed3" containerName="revoke-keys"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.199985    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" containerName="main"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.199995    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" containerName="main"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200006    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200013    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200035    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200043    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200063    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" containerName="main"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200070    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" containerName="main"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200078    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e099851-c987-435f-84d5-4519576c4614" containerName="main"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200085    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e099851-c987-435f-84d5-4519576c4614" containerName="main"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200102    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="615e2500-2db0-4d7c-9a69-69348178f6a6" containerName="main"
Sep 09 12:35:37.200149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200111    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="615e2500-2db0-4d7c-9a69-69348178f6a6" containerName="main"
Sep 09 12:35:37.200845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200209    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f31d601-a6ce-4d90-bcbe-3698ec963ed3" containerName="revoke-keys"
Sep 09 12:35:37.200845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200224    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3453ea77-0da0-4e6a-a413-8ddb50448697" containerName="payload-pre-processing"
Sep 09 12:35:37.200845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200236    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c90e5774-cb91-45a7-ae31-3a1a8c2a48e6" containerName="main"
Sep 09 12:35:37.200845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200245    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="66c97af3-b0ff-4dfb-9c86-32a5b3a2dcf6" containerName="main"
Sep 09 12:35:37.200845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200252    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="615e2500-2db0-4d7c-9a69-69348178f6a6" containerName="main"
Sep 09 12:35:37.200845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200263    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy"
Sep 09 12:35:37.200845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200271    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="76595636-4e31-433e-86de-3c4df5e8ed8a" containerName="istio-proxy"
Sep 09 12:35:37.200845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.200280    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e099851-c987-435f-84d5-4519576c4614" containerName="main"
Sep 09 12:35:37.204631 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.204612    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.207146 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.207120    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-bc0270\"/\"e2e-conflict-a-67a70fd9-kserve-self-signed-certs\""
Sep 09 12:35:37.213819 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.213793    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"]
Sep 09 12:35:37.271341 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.271303    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tmp-dir\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.271531 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.271349    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-home\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.271531 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.271457    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-dshm\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.271661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.271542    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-model-cache\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.271661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.271580    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tls-certs\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.271661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.271606    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rftc\" (UniqueName: \"kubernetes.io/projected/cacd353a-33d8-4497-a2a0-bdf105fcd58f-kube-api-access-7rftc\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.373087 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.373049    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-dshm\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.373267 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.373156    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-model-cache\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.373267 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.373222    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tls-certs\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.373267 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.373250    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rftc\" (UniqueName: \"kubernetes.io/projected/cacd353a-33d8-4497-a2a0-bdf105fcd58f-kube-api-access-7rftc\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.373424 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.373315    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tmp-dir\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.373424 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.373353    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-home\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.373583 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.373550    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-model-cache\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.373740 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.373659    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-home\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.373840 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.373820    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tmp-dir\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.375690 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.375664    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-dshm\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.375856 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.375840    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tls-certs\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.384184 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.384159    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rftc\" (UniqueName: \"kubernetes.io/projected/cacd353a-33d8-4497-a2a0-bdf105fcd58f-kube-api-access-7rftc\") pod \"e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") " pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.515248 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.515209    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:37.791678 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.791651    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"]
Sep 09 12:35:37.793794 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:35:37.793768    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcacd353a_33d8_4497_a2a0_bdf105fcd58f.slice/crio-a966fd2a4896a22b227812b14471339d85d65053953f655d439d31dc0ac57698 WatchSource:0}: Error finding container a966fd2a4896a22b227812b14471339d85d65053953f655d439d31dc0ac57698: Status 404 returned error can't find the container with id a966fd2a4896a22b227812b14471339d85d65053953f655d439d31dc0ac57698
Sep 09 12:35:37.849757 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:37.849680    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8" event={"ID":"cacd353a-33d8-4497-a2a0-bdf105fcd58f","Type":"ContainerStarted","Data":"a966fd2a4896a22b227812b14471339d85d65053953f655d439d31dc0ac57698"}
Sep 09 12:35:38.857906 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:38.857865    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8" event={"ID":"cacd353a-33d8-4497-a2a0-bdf105fcd58f","Type":"ContainerStarted","Data":"bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940"}
Sep 09 12:35:38.874774 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:38.874719    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8" podStartSLOduration=1.8746794599999999 podStartE2EDuration="1.87467946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:35:38.873773305 +0000 UTC m=+1483.515392511" watchObservedRunningTime="2026-09-09 12:35:38.87467946 +0000 UTC m=+1483.516298664"
Sep 09 12:35:47.516111 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:47.516070    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:47.516620 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:47.516131    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:47.529539 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:47.529512    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:47.926394 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:47.926304    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:35:56.093941 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:35:56.093905    2585 scope.go:117] "RemoveContainer" containerID="5adf35c769a18cc304314658b127074571a506fa92dfd7fbcd55b525cff67abc"
Sep 09 12:36:39.494182 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:39.494140    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"]
Sep 09 12:36:41.934134 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:41.934097    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"]
Sep 09 12:36:41.937444 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:41.937422    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:41.939829 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:41.939809    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-364341-openshift-default-dockercfg-mzqqt\""
Sep 09 12:36:41.951953 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:41.951923    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"]
Sep 09 12:36:42.055017 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.054973    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-podinfo\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.055221 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.055078    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcnww\" (UniqueName: \"kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-kube-api-access-mcnww\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.055221 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.055132    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-socket\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.055221 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.055159    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33009dc0-0ce1-41f8-b737-a73b430603a0-istiod-ca-cert\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.055364 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.055236    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-credential-socket\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.055364 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.055266    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-token\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.055364 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.055334    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-data\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.055501 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.055379    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-envoy\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.055501 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.055408    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-certs\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157051 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157009    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-credential-socket\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157051 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157053    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-token\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157305 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157093    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-data\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157305 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157120    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-envoy\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157305 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157138    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-certs\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157305 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157226    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-podinfo\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157515 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157336    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcnww\" (UniqueName: \"kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-kube-api-access-mcnww\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157515 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157369    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-socket\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157515 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157396    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33009dc0-0ce1-41f8-b737-a73b430603a0-istiod-ca-cert\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157515 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157471    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-credential-socket\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157515 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157482    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-data\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157793 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157585    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-certs\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.157793 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.157740    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-socket\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.158096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.158072    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33009dc0-0ce1-41f8-b737-a73b430603a0-istiod-ca-cert\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.159628 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.159597    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-envoy\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.160078 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.160061    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-podinfo\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.165685 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.165657    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-token\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.166027 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.166005    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcnww\" (UniqueName: \"kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-kube-api-access-mcnww\") pod \"e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") " pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.249808 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.249718    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:42.529944 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.529915    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"]
Sep 09 12:36:42.532565 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:36:42.532534    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33009dc0_0ce1_41f8_b737_a73b430603a0.slice/crio-68718719e7d6efb5562de0e69629b89d29e4985aaac9732863945344cf0d852a WatchSource:0}: Error finding container 68718719e7d6efb5562de0e69629b89d29e4985aaac9732863945344cf0d852a: Status 404 returned error can't find the container with id 68718719e7d6efb5562de0e69629b89d29e4985aaac9732863945344cf0d852a
Sep 09 12:36:42.534898 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.534866    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:36:42.534980 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.534933    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:36:42.534980 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:42.534969    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:36:43.233413 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:43.233376    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" event={"ID":"33009dc0-0ce1-41f8-b737-a73b430603a0","Type":"ContainerStarted","Data":"21ea6ff79202acf8a467167a53b6206e3da2ec76fe5712b879fdeb507cea5b69"}
Sep 09 12:36:43.233829 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:43.233433    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" event={"ID":"33009dc0-0ce1-41f8-b737-a73b430603a0","Type":"ContainerStarted","Data":"68718719e7d6efb5562de0e69629b89d29e4985aaac9732863945344cf0d852a"}
Sep 09 12:36:43.250389 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:43.250356    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:43.253986 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:43.253932    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" podStartSLOduration=2.2539162360000002 podStartE2EDuration="2.253916236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:36:43.253273893 +0000 UTC m=+1547.894893100" watchObservedRunningTime="2026-09-09 12:36:43.253916236 +0000 UTC m=+1547.895535443"
Sep 09 12:36:43.257281 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:43.257256    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:44.238973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:44.238945    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:44.240002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:44.239983    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:36:47.269823 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.269774    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"]
Sep 09 12:36:47.273776 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.273754    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:36:47.276457 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.276433    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-364341-dockercfg-r862w\""
Sep 09 12:36:47.276616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.276545    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-364341\""
Sep 09 12:36:47.283447 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.283409    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"]
Sep 09 12:36:47.417101 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.417053    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kck52\" (UniqueName: \"kubernetes.io/projected/c0f97116-5015-4e12-bdc5-97e87f74fe0d-kube-api-access-kck52\") pod \"payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6\" (UID: \"c0f97116-5015-4e12-bdc5-97e87f74fe0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:36:47.417101 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.417104    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f97116-5015-4e12-bdc5-97e87f74fe0d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6\" (UID: \"c0f97116-5015-4e12-bdc5-97e87f74fe0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:36:47.518515 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.518474    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kck52\" (UniqueName: \"kubernetes.io/projected/c0f97116-5015-4e12-bdc5-97e87f74fe0d-kube-api-access-kck52\") pod \"payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6\" (UID: \"c0f97116-5015-4e12-bdc5-97e87f74fe0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:36:47.518515 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.518511    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f97116-5015-4e12-bdc5-97e87f74fe0d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6\" (UID: \"c0f97116-5015-4e12-bdc5-97e87f74fe0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:36:47.519270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.519248    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f97116-5015-4e12-bdc5-97e87f74fe0d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6\" (UID: \"c0f97116-5015-4e12-bdc5-97e87f74fe0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:36:47.527908 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.527838    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kck52\" (UniqueName: \"kubernetes.io/projected/c0f97116-5015-4e12-bdc5-97e87f74fe0d-kube-api-access-kck52\") pod \"payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6\" (UID: \"c0f97116-5015-4e12-bdc5-97e87f74fe0d\") " pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:36:47.588925 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.588889    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:36:47.870368 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:47.870344    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"]
Sep 09 12:36:47.872466 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:36:47.872438    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f97116_5015_4e12_bdc5_97e87f74fe0d.slice/crio-b27f85826fa3021db57cb6d619543e96c10025c7e1a510a731888447934e6862 WatchSource:0}: Error finding container b27f85826fa3021db57cb6d619543e96c10025c7e1a510a731888447934e6862: Status 404 returned error can't find the container with id b27f85826fa3021db57cb6d619543e96c10025c7e1a510a731888447934e6862
Sep 09 12:36:48.264123 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:48.264084    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6" event={"ID":"c0f97116-5015-4e12-bdc5-97e87f74fe0d","Type":"ContainerStarted","Data":"c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3"}
Sep 09 12:36:48.264318 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:48.264148    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6" event={"ID":"c0f97116-5015-4e12-bdc5-97e87f74fe0d","Type":"ContainerStarted","Data":"b27f85826fa3021db57cb6d619543e96c10025c7e1a510a731888447934e6862"}
Sep 09 12:36:48.264318 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:48.264205    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:36:48.284049 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:48.283996    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6" podStartSLOduration=1.283980678 podStartE2EDuration="1.283980678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:36:48.281422913 +0000 UTC m=+1552.923042119" watchObservedRunningTime="2026-09-09 12:36:48.283980678 +0000 UTC m=+1552.925599881"
Sep 09 12:36:54.515569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.515513    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8" podUID="cacd353a-33d8-4497-a2a0-bdf105fcd58f" containerName="main" containerID="cri-o://bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940" gracePeriod=45
Sep 09 12:36:54.780420 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.780353    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:36:54.903939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.903898    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-dshm\") pod \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") "
Sep 09 12:36:54.903939 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.903942    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rftc\" (UniqueName: \"kubernetes.io/projected/cacd353a-33d8-4497-a2a0-bdf105fcd58f-kube-api-access-7rftc\") pod \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") "
Sep 09 12:36:54.904194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.903986    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-model-cache\") pod \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") "
Sep 09 12:36:54.904194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.904055    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-home\") pod \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") "
Sep 09 12:36:54.904194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.904139    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tmp-dir\") pod \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") "
Sep 09 12:36:54.904194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.904188    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tls-certs\") pod \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\" (UID: \"cacd353a-33d8-4497-a2a0-bdf105fcd58f\") "
Sep 09 12:36:54.904397 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.904321    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-model-cache" (OuterVolumeSpecName: "model-cache") pod "cacd353a-33d8-4497-a2a0-bdf105fcd58f" (UID: "cacd353a-33d8-4497-a2a0-bdf105fcd58f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:54.904492 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.904403    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-home" (OuterVolumeSpecName: "home") pod "cacd353a-33d8-4497-a2a0-bdf105fcd58f" (UID: "cacd353a-33d8-4497-a2a0-bdf105fcd58f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:54.904492 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.904415    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cacd353a-33d8-4497-a2a0-bdf105fcd58f" (UID: "cacd353a-33d8-4497-a2a0-bdf105fcd58f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:54.904613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.904587    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:36:54.904747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.904637    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:36:54.904824 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.904786    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:36:54.906443 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.906416    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cacd353a-33d8-4497-a2a0-bdf105fcd58f" (UID: "cacd353a-33d8-4497-a2a0-bdf105fcd58f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:36:54.906443 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.906421    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-dshm" (OuterVolumeSpecName: "dshm") pod "cacd353a-33d8-4497-a2a0-bdf105fcd58f" (UID: "cacd353a-33d8-4497-a2a0-bdf105fcd58f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:54.906562 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:54.906422    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cacd353a-33d8-4497-a2a0-bdf105fcd58f-kube-api-access-7rftc" (OuterVolumeSpecName: "kube-api-access-7rftc") pod "cacd353a-33d8-4497-a2a0-bdf105fcd58f" (UID: "cacd353a-33d8-4497-a2a0-bdf105fcd58f"). InnerVolumeSpecName "kube-api-access-7rftc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:36:55.005971 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.005940    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cacd353a-33d8-4497-a2a0-bdf105fcd58f-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:36:55.005971 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.005970    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rftc\" (UniqueName: \"kubernetes.io/projected/cacd353a-33d8-4497-a2a0-bdf105fcd58f-kube-api-access-7rftc\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:36:55.006166 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.005984    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cacd353a-33d8-4497-a2a0-bdf105fcd58f-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:36:55.306400 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.306366    2585 generic.go:356] "Generic (PLEG): container finished" podID="cacd353a-33d8-4497-a2a0-bdf105fcd58f" containerID="bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940" exitCode=0
Sep 09 12:36:55.306599 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.306432    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"
Sep 09 12:36:55.306599 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.306470    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8" event={"ID":"cacd353a-33d8-4497-a2a0-bdf105fcd58f","Type":"ContainerDied","Data":"bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940"}
Sep 09 12:36:55.306599 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.306512    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8" event={"ID":"cacd353a-33d8-4497-a2a0-bdf105fcd58f","Type":"ContainerDied","Data":"a966fd2a4896a22b227812b14471339d85d65053953f655d439d31dc0ac57698"}
Sep 09 12:36:55.306599 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.306529    2585 scope.go:117] "RemoveContainer" containerID="bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940"
Sep 09 12:36:55.318200 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.318179    2585 scope.go:117] "RemoveContainer" containerID="bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940"
Sep 09 12:36:55.318539 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:36:55.318515    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940\": container with ID starting with bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940 not found: ID does not exist" containerID="bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940"
Sep 09 12:36:55.318636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.318542    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940"} err="failed to get container status \"bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940\": rpc error: code = NotFound desc = could not find container \"bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940\": container with ID starting with bc90c99edde75bb67603fadbb53b5bbd050659ec8fad2f615a57e5c0e2bdc940 not found: ID does not exist"
Sep 09 12:36:55.331819 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.331794    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"]
Sep 09 12:36:55.335926 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.335903    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/e2e-conflict-a-67a70fd9-kserve-797b86898c-fllh8"]
Sep 09 12:36:55.906569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:55.906532    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cacd353a-33d8-4497-a2a0-bdf105fcd58f" path="/var/lib/kubelet/pods/cacd353a-33d8-4497-a2a0-bdf105fcd58f/volumes"
Sep 09 12:36:56.137569 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:56.137536    2585 scope.go:117] "RemoveContainer" containerID="02ad02f9c34c7b5170f10d8f69ccddf6f970e9e66fa45249e20d1f996a2b3bfd"
Sep 09 12:36:56.146651 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:56.146627    2585 scope.go:117] "RemoveContainer" containerID="02333559a18053318b139de6a817978018ea1bc4916688133e26600123cc2e7e"
Sep 09 12:36:59.272732 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:36:59.272669    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:37:11.116373 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.116338    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"]
Sep 09 12:37:11.116884 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.116870    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cacd353a-33d8-4497-a2a0-bdf105fcd58f" containerName="main"
Sep 09 12:37:11.116933 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.116886    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cacd353a-33d8-4497-a2a0-bdf105fcd58f" containerName="main"
Sep 09 12:37:11.116974 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.116964    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cacd353a-33d8-4497-a2a0-bdf105fcd58f" containerName="main"
Sep 09 12:37:11.122275 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.122251    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.124622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.124598    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-59ce1d-openshift-default-dockercfg-hmg5j\""
Sep 09 12:37:11.134453 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.134424    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"]
Sep 09 12:37:11.186236 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.186194    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-data\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.186442 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.186256    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-certs\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.186442 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.186318    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-credential-socket\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.186442 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.186394    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istiod-ca-cert\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.186587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.186461    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-podinfo\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.186587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.186543    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-socket\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.186587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.186567    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-envoy\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.186587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.186584    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-token\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.186782 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.186608    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psdqg\" (UniqueName: \"kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-kube-api-access-psdqg\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.287416 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.287367    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istiod-ca-cert\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.287602 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.287559    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-podinfo\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.287664 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.287641    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-socket\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.287737 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.287672    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-envoy\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.287737 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.287723    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-token\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.287831 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.287754    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psdqg\" (UniqueName: \"kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-kube-api-access-psdqg\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.287831 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.287792    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-data\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.287896 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.287845    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-certs\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.287933 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.287905    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-credential-socket\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.288105 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.288079    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-socket\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.288171 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.288153    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-data\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.288446 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.288422    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-credential-socket\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.288754 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.288734    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istiod-ca-cert\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.288845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.288751    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-certs\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.290360 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.290332    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-envoy\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.291118 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.291098    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-podinfo\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.296432 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.296398    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-token\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.296611 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.296590    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psdqg\" (UniqueName: \"kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-kube-api-access-psdqg\") pod \"e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") " pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.437219 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.437131    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:11.741655 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.741620    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"]
Sep 09 12:37:11.743466 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:37:11.743444    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc51cb4_0bc5_41ce_8949_fe743f51e760.slice/crio-af86389a9ee5302f2e19018e12aba1ff6ddbef4af28074f87b6ea4c11f403a6d WatchSource:0}: Error finding container af86389a9ee5302f2e19018e12aba1ff6ddbef4af28074f87b6ea4c11f403a6d: Status 404 returned error can't find the container with id af86389a9ee5302f2e19018e12aba1ff6ddbef4af28074f87b6ea4c11f403a6d
Sep 09 12:37:11.745678 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.745645    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:37:11.745807 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.745743    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:37:11.745807 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:11.745792    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:37:12.407626 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:12.407592    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" event={"ID":"0bc51cb4-0bc5-41ce-8949-fe743f51e760","Type":"ContainerStarted","Data":"7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b"}
Sep 09 12:37:12.408075 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:12.407647    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" event={"ID":"0bc51cb4-0bc5-41ce-8949-fe743f51e760","Type":"ContainerStarted","Data":"af86389a9ee5302f2e19018e12aba1ff6ddbef4af28074f87b6ea4c11f403a6d"}
Sep 09 12:37:12.429996 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:12.429938    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" podStartSLOduration=1.42992027 podStartE2EDuration="1.42992027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:37:12.429035457 +0000 UTC m=+1577.070654662" watchObservedRunningTime="2026-09-09 12:37:12.42992027 +0000 UTC m=+1577.071539468"
Sep 09 12:37:12.437405 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:12.437373    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:12.442645 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:12.442621    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:13.413627 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:13.413593    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:13.414853 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:13.414832    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:19.449815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:19.449678    2585 generic.go:356] "Generic (PLEG): container finished" podID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerID="21ea6ff79202acf8a467167a53b6206e3da2ec76fe5712b879fdeb507cea5b69" exitCode=0
Sep 09 12:37:19.449815 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:19.449776    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" event={"ID":"33009dc0-0ce1-41f8-b737-a73b430603a0","Type":"ContainerDied","Data":"21ea6ff79202acf8a467167a53b6206e3da2ec76fe5712b879fdeb507cea5b69"}
Sep 09 12:37:19.450335 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:19.450316    2585 scope.go:117] "RemoveContainer" containerID="21ea6ff79202acf8a467167a53b6206e3da2ec76fe5712b879fdeb507cea5b69"
Sep 09 12:37:19.451629 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:19.451595    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:37:19.451748 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:19.451669    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:37:19.451748 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:19.451733    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:37:20.461991 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:20.461940    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" event={"ID":"33009dc0-0ce1-41f8-b737-a73b430603a0","Type":"ContainerStarted","Data":"eb8ce6a0f74d339aef6114af18c9e04dfad3255eead3ddabb41b960bd48dd22e"}
Sep 09 12:37:21.250935 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:21.250886    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:37:21.251203 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:21.251173    2585 patch_prober.go:28] interesting pod/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:21.251285 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:21.251243    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 12:37:22.250778 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:22.250689    2585 patch_prober.go:28] interesting pod/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:22.251319 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:22.250809    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 12:37:23.251132 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:23.251091    2585 patch_prober.go:28] interesting pod/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:23.251743 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:23.251683    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 12:37:24.250740 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:24.250671    2585 patch_prober.go:28] interesting pod/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:24.250971 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:24.250776    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 12:37:25.250544 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:25.250505    2585 patch_prober.go:28] interesting pod/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:25.251185 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:25.251149    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 12:37:26.255076 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:26.255043    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:37:26.255545 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:26.255386    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:37:26.256235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:26.256216    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:37:41.603189 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:41.603151    2585 generic.go:356] "Generic (PLEG): container finished" podID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerID="7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b" exitCode=0
Sep 09 12:37:41.603648 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:41.603195    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" event={"ID":"0bc51cb4-0bc5-41ce-8949-fe743f51e760","Type":"ContainerDied","Data":"7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b"}
Sep 09 12:37:41.603648 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:41.603645    2585 scope.go:117] "RemoveContainer" containerID="7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b"
Sep 09 12:37:41.604922 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:41.604893    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:37:41.604991 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:41.604964    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:37:41.605033 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:41.604996    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:37:42.611255 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:42.611223    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" event={"ID":"0bc51cb4-0bc5-41ce-8949-fe743f51e760","Type":"ContainerStarted","Data":"4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c"}
Sep 09 12:37:42.764636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:42.764591    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"]
Sep 09 12:37:42.768838 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:42.768807    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"
Sep 09 12:37:42.772127 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:42.772096    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:37:42.772454 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:42.772430    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:37:42.780634 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:42.780589    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"]
Sep 09 12:37:42.835121 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:42.835082    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b51c600-694a-49dd-9b06-145c95d10b6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-364341-nmnxz\" (UID: \"1b51c600-694a-49dd-9b06-145c95d10b6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"
Sep 09 12:37:42.935967 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:42.935874    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b51c600-694a-49dd-9b06-145c95d10b6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-364341-nmnxz\" (UID: \"1b51c600-694a-49dd-9b06-145c95d10b6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"
Sep 09 12:37:42.936600 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:42.936576    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b51c600-694a-49dd-9b06-145c95d10b6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-364341-nmnxz\" (UID: \"1b51c600-694a-49dd-9b06-145c95d10b6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"
Sep 09 12:37:43.084951 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:43.084906    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"
Sep 09 12:37:43.365684 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:43.365643    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"]
Sep 09 12:37:43.368636 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:37:43.368602    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b51c600_694a_49dd_9b06_145c95d10b6e.slice/crio-9d7da2b8702eefefde7ab69f0595579d13da0eaf3ce527731c1f6a79e5759565 WatchSource:0}: Error finding container 9d7da2b8702eefefde7ab69f0595579d13da0eaf3ce527731c1f6a79e5759565: Status 404 returned error can't find the container with id 9d7da2b8702eefefde7ab69f0595579d13da0eaf3ce527731c1f6a79e5759565
Sep 09 12:37:43.414408 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:43.414369    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:43.437575 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:43.437536    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:43.438948 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:43.438925    2585 patch_prober.go:28] interesting pod/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 09 12:37:43.439006 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:43.438974    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 09 12:37:43.620101 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:43.620003    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz" event={"ID":"1b51c600-694a-49dd-9b06-145c95d10b6e","Type":"ContainerStarted","Data":"9d7da2b8702eefefde7ab69f0595579d13da0eaf3ce527731c1f6a79e5759565"}
Sep 09 12:37:44.438121 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:44.438091    2585 patch_prober.go:28] interesting pod/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 09 12:37:44.438280 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:44.438144    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 09 12:37:44.627397 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:44.627311    2585 generic.go:356] "Generic (PLEG): container finished" podID="1b51c600-694a-49dd-9b06-145c95d10b6e" containerID="540f486927a5e1b8a0b5016be1b86c788d62ce95db051a3076e8c5d382620285" exitCode=0
Sep 09 12:37:44.627781 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:44.627403    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz" event={"ID":"1b51c600-694a-49dd-9b06-145c95d10b6e","Type":"ContainerDied","Data":"540f486927a5e1b8a0b5016be1b86c788d62ce95db051a3076e8c5d382620285"}
Sep 09 12:37:45.437440 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:45.437403    2585 patch_prober.go:28] interesting pod/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 09 12:37:45.437631 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:45.437460    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 09 12:37:45.817475 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:45.817450    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"
Sep 09 12:37:45.974781 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:45.974216    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b51c600-694a-49dd-9b06-145c95d10b6e-maas-api-service-ca\") pod \"1b51c600-694a-49dd-9b06-145c95d10b6e\" (UID: \"1b51c600-694a-49dd-9b06-145c95d10b6e\") "
Sep 09 12:37:45.974781 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:45.974663    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b51c600-694a-49dd-9b06-145c95d10b6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b51c600-694a-49dd-9b06-145c95d10b6e" (UID: "1b51c600-694a-49dd-9b06-145c95d10b6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:37:46.075908 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:46.075861    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b51c600-694a-49dd-9b06-145c95d10b6e-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:37:46.438506 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:46.438469    2585 patch_prober.go:28] interesting pod/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 09 12:37:46.438688 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:46.438531    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 09 12:37:46.640175 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:46.640140    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz" event={"ID":"1b51c600-694a-49dd-9b06-145c95d10b6e","Type":"ContainerDied","Data":"9d7da2b8702eefefde7ab69f0595579d13da0eaf3ce527731c1f6a79e5759565"}
Sep 09 12:37:46.640175 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:46.640159    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"
Sep 09 12:37:46.640406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:46.640196    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d7da2b8702eefefde7ab69f0595579d13da0eaf3ce527731c1f6a79e5759565"
Sep 09 12:37:47.438113 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:47.438068    2585 patch_prober.go:28] interesting pod/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 09 12:37:47.438595 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:47.438127    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 09 12:37:48.441523 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:48.441492    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:48.442885 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:48.442864    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:37:56.176104 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:56.176073    2585 scope.go:117] "RemoveContainer" containerID="9f932a6e1bf09701bcf56d1bbc6056f4a9f6ea68547ef475fc8f62373ee01b98"
Sep 09 12:37:58.432987 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.432952    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"]
Sep 09 12:37:58.433404 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.433180    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6" podUID="c0f97116-5015-4e12-bdc5-97e87f74fe0d" containerName="payload-processing" containerID="cri-o://c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3" gracePeriod=30
Sep 09 12:37:58.640977 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.640894    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"]
Sep 09 12:37:58.647093 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.647063    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-364341-nmnxz"]
Sep 09 12:37:58.695785 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.695718    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:37:58.715844 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.715800    2585 generic.go:356] "Generic (PLEG): container finished" podID="c0f97116-5015-4e12-bdc5-97e87f74fe0d" containerID="c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3" exitCode=0
Sep 09 12:37:58.716030 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.715873    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6" event={"ID":"c0f97116-5015-4e12-bdc5-97e87f74fe0d","Type":"ContainerDied","Data":"c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3"}
Sep 09 12:37:58.716030 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.715921    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6" event={"ID":"c0f97116-5015-4e12-bdc5-97e87f74fe0d","Type":"ContainerDied","Data":"b27f85826fa3021db57cb6d619543e96c10025c7e1a510a731888447934e6862"}
Sep 09 12:37:58.716030 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.715953    2585 scope.go:117] "RemoveContainer" containerID="c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3"
Sep 09 12:37:58.716202 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.716117    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"
Sep 09 12:37:58.726202 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.726182    2585 scope.go:117] "RemoveContainer" containerID="c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3"
Sep 09 12:37:58.726532 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:37:58.726507    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3\": container with ID starting with c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3 not found: ID does not exist" containerID="c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3"
Sep 09 12:37:58.726625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.726547    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3"} err="failed to get container status \"c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3\": rpc error: code = NotFound desc = could not find container \"c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3\": container with ID starting with c6ac65bb1c8d533d3ac0a5dfdb892d953428836560507c9b8a420e07682c7bd3 not found: ID does not exist"
Sep 09 12:37:58.814235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.814200    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kck52\" (UniqueName: \"kubernetes.io/projected/c0f97116-5015-4e12-bdc5-97e87f74fe0d-kube-api-access-kck52\") pod \"c0f97116-5015-4e12-bdc5-97e87f74fe0d\" (UID: \"c0f97116-5015-4e12-bdc5-97e87f74fe0d\") "
Sep 09 12:37:58.814235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.814239    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f97116-5015-4e12-bdc5-97e87f74fe0d-plugins-config-volume\") pod \"c0f97116-5015-4e12-bdc5-97e87f74fe0d\" (UID: \"c0f97116-5015-4e12-bdc5-97e87f74fe0d\") "
Sep 09 12:37:58.814647 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.814625    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f97116-5015-4e12-bdc5-97e87f74fe0d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c0f97116-5015-4e12-bdc5-97e87f74fe0d" (UID: "c0f97116-5015-4e12-bdc5-97e87f74fe0d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:37:58.816523 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.816493    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f97116-5015-4e12-bdc5-97e87f74fe0d-kube-api-access-kck52" (OuterVolumeSpecName: "kube-api-access-kck52") pod "c0f97116-5015-4e12-bdc5-97e87f74fe0d" (UID: "c0f97116-5015-4e12-bdc5-97e87f74fe0d"). InnerVolumeSpecName "kube-api-access-kck52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:37:58.915575 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.915537    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kck52\" (UniqueName: \"kubernetes.io/projected/c0f97116-5015-4e12-bdc5-97e87f74fe0d-kube-api-access-kck52\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:37:58.915803 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:58.915645    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f97116-5015-4e12-bdc5-97e87f74fe0d-plugins-config-volume\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:37:59.039780 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:59.039741    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"]
Sep 09 12:37:59.041970 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:59.041941    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-364341-84cfbf64b9-ff8s6"]
Sep 09 12:37:59.900711 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:59.900665    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b51c600-694a-49dd-9b06-145c95d10b6e" path="/var/lib/kubelet/pods/1b51c600-694a-49dd-9b06-145c95d10b6e/volumes"
Sep 09 12:37:59.901087 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:37:59.901074    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f97116-5015-4e12-bdc5-97e87f74fe0d" path="/var/lib/kubelet/pods/c0f97116-5015-4e12-bdc5-97e87f74fe0d/volumes"
Sep 09 12:38:05.667931 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:05.667890    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"]
Sep 09 12:38:05.668397 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:05.668150    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy" containerID="cri-o://eb8ce6a0f74d339aef6114af18c9e04dfad3255eead3ddabb41b960bd48dd22e" gracePeriod=30
Sep 09 12:38:06.075690 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.075646    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"]
Sep 09 12:38:06.076194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.076177    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b51c600-694a-49dd-9b06-145c95d10b6e" containerName="revoke-keys"
Sep 09 12:38:06.076263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.076195    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b51c600-694a-49dd-9b06-145c95d10b6e" containerName="revoke-keys"
Sep 09 12:38:06.076263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.076223    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f97116-5015-4e12-bdc5-97e87f74fe0d" containerName="payload-processing"
Sep 09 12:38:06.076263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.076229    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f97116-5015-4e12-bdc5-97e87f74fe0d" containerName="payload-processing"
Sep 09 12:38:06.076375 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.076290    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b51c600-694a-49dd-9b06-145c95d10b6e" containerName="revoke-keys"
Sep 09 12:38:06.076375 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.076310    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f97116-5015-4e12-bdc5-97e87f74fe0d" containerName="payload-processing"
Sep 09 12:38:06.080847 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.080830    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"
Sep 09 12:38:06.082973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.082948    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:38:06.083094 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.082989    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:38:06.087155 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.087123    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"]
Sep 09 12:38:06.193770 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.193734    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb524ce1-d302-4961-a356-484ad3192e7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs\" (UID: \"eb524ce1-d302-4961-a356-484ad3192e7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"
Sep 09 12:38:06.295247 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.295205    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb524ce1-d302-4961-a356-484ad3192e7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs\" (UID: \"eb524ce1-d302-4961-a356-484ad3192e7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"
Sep 09 12:38:06.296382 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.296351    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb524ce1-d302-4961-a356-484ad3192e7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs\" (UID: \"eb524ce1-d302-4961-a356-484ad3192e7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"
Sep 09 12:38:06.394661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.394571    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"
Sep 09 12:38:06.678585 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.678555    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"]
Sep 09 12:38:06.681351 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:38:06.681322    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb524ce1_d302_4961_a356_484ad3192e7d.slice/crio-2b76510ff83ccd79603459f1c8286dcbdf53388524be6c269fb109a21c252ddb WatchSource:0}: Error finding container 2b76510ff83ccd79603459f1c8286dcbdf53388524be6c269fb109a21c252ddb: Status 404 returned error can't find the container with id 2b76510ff83ccd79603459f1c8286dcbdf53388524be6c269fb109a21c252ddb
Sep 09 12:38:06.765440 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:06.765404    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs" event={"ID":"eb524ce1-d302-4961-a356-484ad3192e7d","Type":"ContainerStarted","Data":"2b76510ff83ccd79603459f1c8286dcbdf53388524be6c269fb109a21c252ddb"}
Sep 09 12:38:07.772208 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:07.772174    2585 generic.go:356] "Generic (PLEG): container finished" podID="eb524ce1-d302-4961-a356-484ad3192e7d" containerID="a02fb4e4cd57f44deb45fb9a0e94c8c5e8173da9ea7b4d650e0ca59db12ed055" exitCode=0
Sep 09 12:38:07.772606 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:07.772263    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs" event={"ID":"eb524ce1-d302-4961-a356-484ad3192e7d","Type":"ContainerDied","Data":"a02fb4e4cd57f44deb45fb9a0e94c8c5e8173da9ea7b4d650e0ca59db12ed055"}
Sep 09 12:38:08.917311 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:08.917282    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"
Sep 09 12:38:09.024901 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:09.024867    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb524ce1-d302-4961-a356-484ad3192e7d-maas-api-service-ca\") pod \"eb524ce1-d302-4961-a356-484ad3192e7d\" (UID: \"eb524ce1-d302-4961-a356-484ad3192e7d\") "
Sep 09 12:38:09.025255 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:09.025231    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb524ce1-d302-4961-a356-484ad3192e7d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb524ce1-d302-4961-a356-484ad3192e7d" (UID: "eb524ce1-d302-4961-a356-484ad3192e7d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:09.126233 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:09.126150    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb524ce1-d302-4961-a356-484ad3192e7d-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:09.785747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:09.785710    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs" event={"ID":"eb524ce1-d302-4961-a356-484ad3192e7d","Type":"ContainerDied","Data":"2b76510ff83ccd79603459f1c8286dcbdf53388524be6c269fb109a21c252ddb"}
Sep 09 12:38:09.785747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:09.785730    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"
Sep 09 12:38:09.785962 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:09.785759    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b76510ff83ccd79603459f1c8286dcbdf53388524be6c269fb109a21c252ddb"
Sep 09 12:38:10.794081 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:10.794053    2585 generic.go:356] "Generic (PLEG): container finished" podID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerID="eb8ce6a0f74d339aef6114af18c9e04dfad3255eead3ddabb41b960bd48dd22e" exitCode=0
Sep 09 12:38:10.794434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:10.794133    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" event={"ID":"33009dc0-0ce1-41f8-b737-a73b430603a0","Type":"ContainerDied","Data":"eb8ce6a0f74d339aef6114af18c9e04dfad3255eead3ddabb41b960bd48dd22e"}
Sep 09 12:38:10.794434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:10.794213    2585 scope.go:117] "RemoveContainer" containerID="21ea6ff79202acf8a467167a53b6206e3da2ec76fe5712b879fdeb507cea5b69"
Sep 09 12:38:10.929828 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:10.929798    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:38:11.047258 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047228    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-socket\") pod \"33009dc0-0ce1-41f8-b737-a73b430603a0\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") "
Sep 09 12:38:11.047452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047264    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-envoy\") pod \"33009dc0-0ce1-41f8-b737-a73b430603a0\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") "
Sep 09 12:38:11.047452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047291    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-token\") pod \"33009dc0-0ce1-41f8-b737-a73b430603a0\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") "
Sep 09 12:38:11.047452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047330    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcnww\" (UniqueName: \"kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-kube-api-access-mcnww\") pod \"33009dc0-0ce1-41f8-b737-a73b430603a0\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") "
Sep 09 12:38:11.047452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047367    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-data\") pod \"33009dc0-0ce1-41f8-b737-a73b430603a0\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") "
Sep 09 12:38:11.047452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047395    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-credential-socket\") pod \"33009dc0-0ce1-41f8-b737-a73b430603a0\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") "
Sep 09 12:38:11.047689 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047531    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-certs\") pod \"33009dc0-0ce1-41f8-b737-a73b430603a0\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") "
Sep 09 12:38:11.047689 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047552    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33009dc0-0ce1-41f8-b737-a73b430603a0" (UID: "33009dc0-0ce1-41f8-b737-a73b430603a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:11.047689 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047617    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-podinfo\") pod \"33009dc0-0ce1-41f8-b737-a73b430603a0\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") "
Sep 09 12:38:11.047689 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047684    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33009dc0-0ce1-41f8-b737-a73b430603a0-istiod-ca-cert\") pod \"33009dc0-0ce1-41f8-b737-a73b430603a0\" (UID: \"33009dc0-0ce1-41f8-b737-a73b430603a0\") "
Sep 09 12:38:11.047927 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047839    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33009dc0-0ce1-41f8-b737-a73b430603a0" (UID: "33009dc0-0ce1-41f8-b737-a73b430603a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:11.047927 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047856    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33009dc0-0ce1-41f8-b737-a73b430603a0" (UID: "33009dc0-0ce1-41f8-b737-a73b430603a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:11.048041 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.047953    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "33009dc0-0ce1-41f8-b737-a73b430603a0" (UID: "33009dc0-0ce1-41f8-b737-a73b430603a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:11.048230 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.048207    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33009dc0-0ce1-41f8-b737-a73b430603a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33009dc0-0ce1-41f8-b737-a73b430603a0" (UID: "33009dc0-0ce1-41f8-b737-a73b430603a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:11.048522 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.048502    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.048803 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.048528    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.048803 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.048544    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.048803 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.048558    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33009dc0-0ce1-41f8-b737-a73b430603a0-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.048803 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.048570    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.049781 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.049760    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33009dc0-0ce1-41f8-b737-a73b430603a0" (UID: "33009dc0-0ce1-41f8-b737-a73b430603a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:11.049968 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.049944    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33009dc0-0ce1-41f8-b737-a73b430603a0" (UID: "33009dc0-0ce1-41f8-b737-a73b430603a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:38:11.050192 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.050172    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "33009dc0-0ce1-41f8-b737-a73b430603a0" (UID: "33009dc0-0ce1-41f8-b737-a73b430603a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:11.050344 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.050326    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-kube-api-access-mcnww" (OuterVolumeSpecName: "kube-api-access-mcnww") pod "33009dc0-0ce1-41f8-b737-a73b430603a0" (UID: "33009dc0-0ce1-41f8-b737-a73b430603a0"). InnerVolumeSpecName "kube-api-access-mcnww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:11.149275 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.149234    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.149275 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.149276    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.149479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.149290    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcnww\" (UniqueName: \"kubernetes.io/projected/33009dc0-0ce1-41f8-b737-a73b430603a0-kube-api-access-mcnww\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.149479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.149308    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33009dc0-0ce1-41f8-b737-a73b430603a0-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.801087 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.801054    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"
Sep 09 12:38:11.801540 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.801054    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj" event={"ID":"33009dc0-0ce1-41f8-b737-a73b430603a0","Type":"ContainerDied","Data":"68718719e7d6efb5562de0e69629b89d29e4985aaac9732863945344cf0d852a"}
Sep 09 12:38:11.801540 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.801189    2585 scope.go:117] "RemoveContainer" containerID="eb8ce6a0f74d339aef6114af18c9e04dfad3255eead3ddabb41b960bd48dd22e"
Sep 09 12:38:11.829435 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.829409    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"]
Sep 09 12:38:11.835823 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.835800    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-364341-openshift-default-774b4b4c54-gfhkj"]
Sep 09 12:38:11.900547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:11.900513    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" path="/var/lib/kubelet/pods/33009dc0-0ce1-41f8-b737-a73b430603a0/volumes"
Sep 09 12:38:21.796049 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:21.796006    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"]
Sep 09 12:38:21.798215 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:21.798176    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-59ce1d-jtfcs"]
Sep 09 12:38:21.902420 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:21.902379    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb524ce1-d302-4961-a356-484ad3192e7d" path="/var/lib/kubelet/pods/eb524ce1-d302-4961-a356-484ad3192e7d/volumes"
Sep 09 12:38:29.304077 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.304040    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"]
Sep 09 12:38:29.304711 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.304277    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy" containerID="cri-o://4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c" gracePeriod=30
Sep 09 12:38:29.707000 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.706913    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"]
Sep 09 12:38:29.707376 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.707363    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb524ce1-d302-4961-a356-484ad3192e7d" containerName="revoke-keys"
Sep 09 12:38:29.707431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.707378    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb524ce1-d302-4961-a356-484ad3192e7d" containerName="revoke-keys"
Sep 09 12:38:29.707431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.707394    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy"
Sep 09 12:38:29.707431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.707400    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy"
Sep 09 12:38:29.707431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.707414    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy"
Sep 09 12:38:29.707431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.707419    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy"
Sep 09 12:38:29.707613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.707486    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb524ce1-d302-4961-a356-484ad3192e7d" containerName="revoke-keys"
Sep 09 12:38:29.707613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.707494    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy"
Sep 09 12:38:29.707613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.707501    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="33009dc0-0ce1-41f8-b737-a73b430603a0" containerName="istio-proxy"
Sep 09 12:38:29.710632 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.710612    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"
Sep 09 12:38:29.713109 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.713087    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:38:29.713163 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.713094    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:38:29.720739 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.720711    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"]
Sep 09 12:38:29.848663 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.848625    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7abc8b55-dd5f-44c4-b63b-a7a51fbde080-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw\" (UID: \"7abc8b55-dd5f-44c4-b63b-a7a51fbde080\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"
Sep 09 12:38:29.949625 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.949582    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7abc8b55-dd5f-44c4-b63b-a7a51fbde080-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw\" (UID: \"7abc8b55-dd5f-44c4-b63b-a7a51fbde080\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"
Sep 09 12:38:29.950235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:29.950219    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7abc8b55-dd5f-44c4-b63b-a7a51fbde080-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw\" (UID: \"7abc8b55-dd5f-44c4-b63b-a7a51fbde080\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"
Sep 09 12:38:30.022366 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:30.022331    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"
Sep 09 12:38:30.302057 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:30.302027    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"]
Sep 09 12:38:30.303903 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:38:30.303860    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7abc8b55_dd5f_44c4_b63b_a7a51fbde080.slice/crio-e58780a856dfe4b732dbbc7616e08e6d714d4297086b7bc5085b020dd99adb6a WatchSource:0}: Error finding container e58780a856dfe4b732dbbc7616e08e6d714d4297086b7bc5085b020dd99adb6a: Status 404 returned error can't find the container with id e58780a856dfe4b732dbbc7616e08e6d714d4297086b7bc5085b020dd99adb6a
Sep 09 12:38:30.912519 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:30.912419    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw" event={"ID":"7abc8b55-dd5f-44c4-b63b-a7a51fbde080","Type":"ContainerStarted","Data":"b827e6763e1a577b408eaae29f3b30fd4b84ac3813c9acdbfb6c73c6a01b0ded"}
Sep 09 12:38:30.912519 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:30.912480    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw" event={"ID":"7abc8b55-dd5f-44c4-b63b-a7a51fbde080","Type":"ContainerStarted","Data":"e58780a856dfe4b732dbbc7616e08e6d714d4297086b7bc5085b020dd99adb6a"}
Sep 09 12:38:30.929561 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:30.929502    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw" podStartSLOduration=1.929485573 podStartE2EDuration="1.929485573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:30.928917264 +0000 UTC m=+1655.570536471" watchObservedRunningTime="2026-09-09 12:38:30.929485573 +0000 UTC m=+1655.571104777"
Sep 09 12:38:31.919018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:31.918985    2585 generic.go:356] "Generic (PLEG): container finished" podID="7abc8b55-dd5f-44c4-b63b-a7a51fbde080" containerID="b827e6763e1a577b408eaae29f3b30fd4b84ac3813c9acdbfb6c73c6a01b0ded" exitCode=0
Sep 09 12:38:31.919420 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:31.919072    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw" event={"ID":"7abc8b55-dd5f-44c4-b63b-a7a51fbde080","Type":"ContainerDied","Data":"b827e6763e1a577b408eaae29f3b30fd4b84ac3813c9acdbfb6c73c6a01b0ded"}
Sep 09 12:38:33.066519 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:33.066493    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"
Sep 09 12:38:33.184493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:33.184402    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7abc8b55-dd5f-44c4-b63b-a7a51fbde080-maas-api-service-ca\") pod \"7abc8b55-dd5f-44c4-b63b-a7a51fbde080\" (UID: \"7abc8b55-dd5f-44c4-b63b-a7a51fbde080\") "
Sep 09 12:38:33.184868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:33.184845    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7abc8b55-dd5f-44c4-b63b-a7a51fbde080-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7abc8b55-dd5f-44c4-b63b-a7a51fbde080" (UID: "7abc8b55-dd5f-44c4-b63b-a7a51fbde080"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:33.184955 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:33.184941    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7abc8b55-dd5f-44c4-b63b-a7a51fbde080-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:33.445215 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:33.445132    2585 patch_prober.go:28] interesting pod/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 12:38:33.445215 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:33.445197    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:38:33.931707 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:33.931670    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"
Sep 09 12:38:33.931877 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:33.931666    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw" event={"ID":"7abc8b55-dd5f-44c4-b63b-a7a51fbde080","Type":"ContainerDied","Data":"e58780a856dfe4b732dbbc7616e08e6d714d4297086b7bc5085b020dd99adb6a"}
Sep 09 12:38:33.931877 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:33.931819    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e58780a856dfe4b732dbbc7616e08e6d714d4297086b7bc5085b020dd99adb6a"
Sep 09 12:38:34.555762 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.555737    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:38:34.599156 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599069    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-podinfo\") pod \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") "
Sep 09 12:38:34.599332 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599160    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-credential-socket\") pod \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") "
Sep 09 12:38:34.599332 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599200    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istiod-ca-cert\") pod \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") "
Sep 09 12:38:34.599332 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599222    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-socket\") pod \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") "
Sep 09 12:38:34.599332 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599246    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psdqg\" (UniqueName: \"kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-kube-api-access-psdqg\") pod \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") "
Sep 09 12:38:34.599332 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599287    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-certs\") pod \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") "
Sep 09 12:38:34.599592 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599339    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-envoy\") pod \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") "
Sep 09 12:38:34.599592 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599364    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-token\") pod \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") "
Sep 09 12:38:34.599592 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599427    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-data\") pod \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\" (UID: \"0bc51cb4-0bc5-41ce-8949-fe743f51e760\") "
Sep 09 12:38:34.599592 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599506    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0bc51cb4-0bc5-41ce-8949-fe743f51e760" (UID: "0bc51cb4-0bc5-41ce-8949-fe743f51e760"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:34.599830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599634    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0bc51cb4-0bc5-41ce-8949-fe743f51e760" (UID: "0bc51cb4-0bc5-41ce-8949-fe743f51e760"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:34.599830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599653    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0bc51cb4-0bc5-41ce-8949-fe743f51e760" (UID: "0bc51cb4-0bc5-41ce-8949-fe743f51e760"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:34.599830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599787    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-data" (OuterVolumeSpecName: "istio-data") pod "0bc51cb4-0bc5-41ce-8949-fe743f51e760" (UID: "0bc51cb4-0bc5-41ce-8949-fe743f51e760"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:34.599830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599811    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0bc51cb4-0bc5-41ce-8949-fe743f51e760" (UID: "0bc51cb4-0bc5-41ce-8949-fe743f51e760"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:34.600028 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599927    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:34.600028 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599945    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:34.600028 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599960    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:34.600028 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599974    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:34.600028 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.599990    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:34.601612 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.601588    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0bc51cb4-0bc5-41ce-8949-fe743f51e760" (UID: "0bc51cb4-0bc5-41ce-8949-fe743f51e760"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:38:34.601835 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.601811    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-kube-api-access-psdqg" (OuterVolumeSpecName: "kube-api-access-psdqg") pod "0bc51cb4-0bc5-41ce-8949-fe743f51e760" (UID: "0bc51cb4-0bc5-41ce-8949-fe743f51e760"). InnerVolumeSpecName "kube-api-access-psdqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:34.601962 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.601945    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-token" (OuterVolumeSpecName: "istio-token") pod "0bc51cb4-0bc5-41ce-8949-fe743f51e760" (UID: "0bc51cb4-0bc5-41ce-8949-fe743f51e760"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:34.602067 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.602052    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0bc51cb4-0bc5-41ce-8949-fe743f51e760" (UID: "0bc51cb4-0bc5-41ce-8949-fe743f51e760"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:34.701308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.701276    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psdqg\" (UniqueName: \"kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-kube-api-access-psdqg\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:34.701308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.701306    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:34.701308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.701315    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:34.701551 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.701322    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bc51cb4-0bc5-41ce-8949-fe743f51e760-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:34.939760 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.939655    2585 generic.go:356] "Generic (PLEG): container finished" podID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerID="4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c" exitCode=0
Sep 09 12:38:34.939760 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.939722    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"
Sep 09 12:38:34.939760 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.939741    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" event={"ID":"0bc51cb4-0bc5-41ce-8949-fe743f51e760","Type":"ContainerDied","Data":"4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c"}
Sep 09 12:38:34.940020 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.939799    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v" event={"ID":"0bc51cb4-0bc5-41ce-8949-fe743f51e760","Type":"ContainerDied","Data":"af86389a9ee5302f2e19018e12aba1ff6ddbef4af28074f87b6ea4c11f403a6d"}
Sep 09 12:38:34.940020 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.939822    2585 scope.go:117] "RemoveContainer" containerID="4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c"
Sep 09 12:38:34.950307 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.950288    2585 scope.go:117] "RemoveContainer" containerID="7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b"
Sep 09 12:38:34.961339 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.961316    2585 scope.go:117] "RemoveContainer" containerID="4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c"
Sep 09 12:38:34.961689 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:38:34.961667    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c\": container with ID starting with 4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c not found: ID does not exist" containerID="4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c"
Sep 09 12:38:34.961808 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.961716    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c"} err="failed to get container status \"4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c\": rpc error: code = NotFound desc = could not find container \"4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c\": container with ID starting with 4eb5b476734bc00eb18023ed4f1a1cffec474810d9e4b37aabf661b5e8f70e8c not found: ID does not exist"
Sep 09 12:38:34.961808 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.961736    2585 scope.go:117] "RemoveContainer" containerID="7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b"
Sep 09 12:38:34.962025 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:38:34.962005    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b\": container with ID starting with 7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b not found: ID does not exist" containerID="7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b"
Sep 09 12:38:34.962069 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.962030    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b"} err="failed to get container status \"7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b\": rpc error: code = NotFound desc = could not find container \"7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b\": container with ID starting with 7602dd9a5528b15830435ddfededfae54c3c58c67f47e27d1676d28d6fe9c42b not found: ID does not exist"
Sep 09 12:38:34.967126 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.967098    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"]
Sep 09 12:38:34.970231 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:34.970205    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-59ce1d-openshift-default-7c7f8db7bd-th59v"]
Sep 09 12:38:35.899786 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:35.899749    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" path="/var/lib/kubelet/pods/0bc51cb4-0bc5-41ce-8949-fe743f51e760/volumes"
Sep 09 12:38:37.470616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.470579    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7884-3c47cc"]
Sep 09 12:38:37.471041 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.471018    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7abc8b55-dd5f-44c4-b63b-a7a51fbde080" containerName="revoke-keys"
Sep 09 12:38:37.471041 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.471029    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7abc8b55-dd5f-44c4-b63b-a7a51fbde080" containerName="revoke-keys"
Sep 09 12:38:37.471041 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.471041    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy"
Sep 09 12:38:37.471157 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.471047    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy"
Sep 09 12:38:37.471157 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.471125    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy"
Sep 09 12:38:37.471157 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.471144    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7abc8b55-dd5f-44c4-b63b-a7a51fbde080" containerName="revoke-keys"
Sep 09 12:38:37.475737 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.475719    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7884-3c47cc"
Sep 09 12:38:37.479247 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.479217    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kwwl8\""
Sep 09 12:38:37.482451 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.482417    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7884-3c47cc"]
Sep 09 12:38:37.528332 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.528279    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlbz7\" (UniqueName: \"kubernetes.io/projected/ed1d2bff-e086-4cc7-8d66-b7c534fc8b22-kube-api-access-vlbz7\") pod \"mt-curl-7884-3c47cc\" (UID: \"ed1d2bff-e086-4cc7-8d66-b7c534fc8b22\") " pod="opendatahub/mt-curl-7884-3c47cc"
Sep 09 12:38:37.629665 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.629622    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlbz7\" (UniqueName: \"kubernetes.io/projected/ed1d2bff-e086-4cc7-8d66-b7c534fc8b22-kube-api-access-vlbz7\") pod \"mt-curl-7884-3c47cc\" (UID: \"ed1d2bff-e086-4cc7-8d66-b7c534fc8b22\") " pod="opendatahub/mt-curl-7884-3c47cc"
Sep 09 12:38:37.640425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.640390    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlbz7\" (UniqueName: \"kubernetes.io/projected/ed1d2bff-e086-4cc7-8d66-b7c534fc8b22-kube-api-access-vlbz7\") pod \"mt-curl-7884-3c47cc\" (UID: \"ed1d2bff-e086-4cc7-8d66-b7c534fc8b22\") " pod="opendatahub/mt-curl-7884-3c47cc"
Sep 09 12:38:37.786337 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:37.786302    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7884-3c47cc"
Sep 09 12:38:38.062823 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:38.062795    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7884-3c47cc"]
Sep 09 12:38:38.064928 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:38:38.064898    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded1d2bff_e086_4cc7_8d66_b7c534fc8b22.slice/crio-13e32b814f004688f99618637128ded12f0d199c252be109f2b0931c6a402312 WatchSource:0}: Error finding container 13e32b814f004688f99618637128ded12f0d199c252be109f2b0931c6a402312: Status 404 returned error can't find the container with id 13e32b814f004688f99618637128ded12f0d199c252be109f2b0931c6a402312
Sep 09 12:38:38.066304 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:38.066288    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:38:38.965848 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:38.965815    2585 generic.go:356] "Generic (PLEG): container finished" podID="ed1d2bff-e086-4cc7-8d66-b7c534fc8b22" containerID="c856465bdb19494afdbb04c9426503570248f448fdddb33fc066c36a7ac53661" exitCode=0
Sep 09 12:38:38.966243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:38.965860    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7884-3c47cc" event={"ID":"ed1d2bff-e086-4cc7-8d66-b7c534fc8b22","Type":"ContainerDied","Data":"c856465bdb19494afdbb04c9426503570248f448fdddb33fc066c36a7ac53661"}
Sep 09 12:38:38.966243 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:38.965890    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7884-3c47cc" event={"ID":"ed1d2bff-e086-4cc7-8d66-b7c534fc8b22","Type":"ContainerStarted","Data":"13e32b814f004688f99618637128ded12f0d199c252be109f2b0931c6a402312"}
Sep 09 12:38:40.114612 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.114589    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7884-3c47cc"
Sep 09 12:38:40.153819 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.153766    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7884-3c47cc_ed1d2bff-e086-4cc7-8d66-b7c534fc8b22/mt-curl-7884-3c47cc/0.log"
Sep 09 12:38:40.157185 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.157147    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlbz7\" (UniqueName: \"kubernetes.io/projected/ed1d2bff-e086-4cc7-8d66-b7c534fc8b22-kube-api-access-vlbz7\") pod \"ed1d2bff-e086-4cc7-8d66-b7c534fc8b22\" (UID: \"ed1d2bff-e086-4cc7-8d66-b7c534fc8b22\") "
Sep 09 12:38:40.159504 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.159474    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed1d2bff-e086-4cc7-8d66-b7c534fc8b22-kube-api-access-vlbz7" (OuterVolumeSpecName: "kube-api-access-vlbz7") pod "ed1d2bff-e086-4cc7-8d66-b7c534fc8b22" (UID: "ed1d2bff-e086-4cc7-8d66-b7c534fc8b22"). InnerVolumeSpecName "kube-api-access-vlbz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:40.180118 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.180088    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7884-3c47cc"]
Sep 09 12:38:40.183369 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.183344    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7884-3c47cc"]
Sep 09 12:38:40.258493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.258459    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlbz7\" (UniqueName: \"kubernetes.io/projected/ed1d2bff-e086-4cc7-8d66-b7c534fc8b22-kube-api-access-vlbz7\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:40.458053 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.458013    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"]
Sep 09 12:38:40.460668 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.460636    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-d1f639-jv5hw"]
Sep 09 12:38:40.979757 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.979723    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13e32b814f004688f99618637128ded12f0d199c252be109f2b0931c6a402312"
Sep 09 12:38:40.979757 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:40.979750    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7884-3c47cc"
Sep 09 12:38:40.990071 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:38:40.990036    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7884-3c47cc\" is forbidden: User \"system:node:ip-10-0-131-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-207.ec2.internal' and this object" podUID="ed1d2bff-e086-4cc7-8d66-b7c534fc8b22" pod="opendatahub/mt-curl-7884-3c47cc"
Sep 09 12:38:41.900873 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:41.900836    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7abc8b55-dd5f-44c4-b63b-a7a51fbde080" path="/var/lib/kubelet/pods/7abc8b55-dd5f-44c4-b63b-a7a51fbde080/volumes"
Sep 09 12:38:41.901263 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:41.901204    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed1d2bff-e086-4cc7-8d66-b7c534fc8b22" path="/var/lib/kubelet/pods/ed1d2bff-e086-4cc7-8d66-b7c534fc8b22/volumes"
Sep 09 12:38:47.793000 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.792954    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"]
Sep 09 12:38:47.793410 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.793397    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy"
Sep 09 12:38:47.793455 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.793411    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy"
Sep 09 12:38:47.793455 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.793420    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed1d2bff-e086-4cc7-8d66-b7c534fc8b22" containerName="mt-curl-7884-3c47cc"
Sep 09 12:38:47.793455 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.793426    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1d2bff-e086-4cc7-8d66-b7c534fc8b22" containerName="mt-curl-7884-3c47cc"
Sep 09 12:38:47.793556 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.793500    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc51cb4-0bc5-41ce-8949-fe743f51e760" containerName="istio-proxy"
Sep 09 12:38:47.793556 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.793511    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed1d2bff-e086-4cc7-8d66-b7c534fc8b22" containerName="mt-curl-7884-3c47cc"
Sep 09 12:38:47.797018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.797002    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"
Sep 09 12:38:47.799393 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.799373    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:38:47.799457 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.799373    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:38:47.804724 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.804599    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"]
Sep 09 12:38:47.836211 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.836178    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de475046-e0b3-47dd-81d9-e7c5e455de80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9\" (UID: \"de475046-e0b3-47dd-81d9-e7c5e455de80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"
Sep 09 12:38:47.937344 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.937303    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de475046-e0b3-47dd-81d9-e7c5e455de80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9\" (UID: \"de475046-e0b3-47dd-81d9-e7c5e455de80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"
Sep 09 12:38:47.937981 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:47.937959    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de475046-e0b3-47dd-81d9-e7c5e455de80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9\" (UID: \"de475046-e0b3-47dd-81d9-e7c5e455de80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"
Sep 09 12:38:48.108889 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:48.108797    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"
Sep 09 12:38:48.400716 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:48.400670    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"]
Sep 09 12:38:48.402965 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:38:48.402936    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde475046_e0b3_47dd_81d9_e7c5e455de80.slice/crio-333a1c16e421228e228ca2ca8478bea7df0774d16ec8ecb1b3648a3d21c3373c WatchSource:0}: Error finding container 333a1c16e421228e228ca2ca8478bea7df0774d16ec8ecb1b3648a3d21c3373c: Status 404 returned error can't find the container with id 333a1c16e421228e228ca2ca8478bea7df0774d16ec8ecb1b3648a3d21c3373c
Sep 09 12:38:49.028513 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:49.028478    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9" event={"ID":"de475046-e0b3-47dd-81d9-e7c5e455de80","Type":"ContainerStarted","Data":"dab7027277d7d16c3429640dd5942396d405894370dd06163071f6cf21a3314b"}
Sep 09 12:38:49.028905 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:49.028533    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9" event={"ID":"de475046-e0b3-47dd-81d9-e7c5e455de80","Type":"ContainerStarted","Data":"333a1c16e421228e228ca2ca8478bea7df0774d16ec8ecb1b3648a3d21c3373c"}
Sep 09 12:38:49.042922 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:49.042860    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9" podStartSLOduration=2.042838955 podStartE2EDuration="2.042838955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:49.042300413 +0000 UTC m=+1673.683919622" watchObservedRunningTime="2026-09-09 12:38:49.042838955 +0000 UTC m=+1673.684458162"
Sep 09 12:38:50.036334 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:50.036298    2585 generic.go:356] "Generic (PLEG): container finished" podID="de475046-e0b3-47dd-81d9-e7c5e455de80" containerID="dab7027277d7d16c3429640dd5942396d405894370dd06163071f6cf21a3314b" exitCode=0
Sep 09 12:38:50.036747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:50.036358    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9" event={"ID":"de475046-e0b3-47dd-81d9-e7c5e455de80","Type":"ContainerDied","Data":"dab7027277d7d16c3429640dd5942396d405894370dd06163071f6cf21a3314b"}
Sep 09 12:38:51.192406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:51.192376    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"
Sep 09 12:38:51.272871 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:51.272834    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de475046-e0b3-47dd-81d9-e7c5e455de80-maas-api-service-ca\") pod \"de475046-e0b3-47dd-81d9-e7c5e455de80\" (UID: \"de475046-e0b3-47dd-81d9-e7c5e455de80\") "
Sep 09 12:38:51.273226 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:51.273201    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de475046-e0b3-47dd-81d9-e7c5e455de80-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de475046-e0b3-47dd-81d9-e7c5e455de80" (UID: "de475046-e0b3-47dd-81d9-e7c5e455de80"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:51.373755 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:51.373636    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de475046-e0b3-47dd-81d9-e7c5e455de80-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:38:52.056301 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:52.056257    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"
Sep 09 12:38:52.056301 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:52.056295    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9" event={"ID":"de475046-e0b3-47dd-81d9-e7c5e455de80","Type":"ContainerDied","Data":"333a1c16e421228e228ca2ca8478bea7df0774d16ec8ecb1b3648a3d21c3373c"}
Sep 09 12:38:52.056494 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:52.056344    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="333a1c16e421228e228ca2ca8478bea7df0774d16ec8ecb1b3648a3d21c3373c"
Sep 09 12:38:55.310596 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.310558    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"]
Sep 09 12:38:55.311082 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.311058    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de475046-e0b3-47dd-81d9-e7c5e455de80" containerName="revoke-keys"
Sep 09 12:38:55.311082 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.311079    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="de475046-e0b3-47dd-81d9-e7c5e455de80" containerName="revoke-keys"
Sep 09 12:38:55.311271 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.311172    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="de475046-e0b3-47dd-81d9-e7c5e455de80" containerName="revoke-keys"
Sep 09 12:38:55.314452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.314436    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.318104 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.318081    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-679f45\"/\"default-dockercfg-skmbs\""
Sep 09 12:38:55.318104 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.318087    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-679f45\"/\"kube-root-ca.crt\""
Sep 09 12:38:55.318362 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.318127    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-679f45\"/\"openshift-service-ca.crt\""
Sep 09 12:38:55.318362 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.318264    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-679f45\"/\"sub-test-model-679f45-kserve-self-signed-certs\""
Sep 09 12:38:55.328252 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.328227    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"]
Sep 09 12:38:55.416473 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.416440    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-home\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.416636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.416491    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g86l4\" (UniqueName: \"kubernetes.io/projected/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-kube-api-access-g86l4\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.416636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.416568    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-model-cache\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.416636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.416625    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tls-certs\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.416775 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.416644    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tmp-dir\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.416775 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.416672    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-dshm\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.517795 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.517757    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-home\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.517983 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.517809    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g86l4\" (UniqueName: \"kubernetes.io/projected/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-kube-api-access-g86l4\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.517983 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.517852    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-model-cache\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.517983 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.517897    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tls-certs\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.517983 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.517922    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tmp-dir\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.518204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.518069    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-dshm\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.518204 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.518180    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-home\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.518324 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.518221    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tmp-dir\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.518324 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.518300    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-model-cache\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.520300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.520265    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-dshm\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.520570 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.520554    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tls-certs\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.531027 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.530997    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g86l4\" (UniqueName: \"kubernetes.io/projected/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-kube-api-access-g86l4\") pod \"sub-test-model-679f45-kserve-79f66695f7-zvt6f\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") " pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.624557 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.624467    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:38:55.908867 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:55.908843    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"]
Sep 09 12:38:55.910723 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:38:55.910662    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a8ebf21_e94d_46e8_bb5f_8b82b51b2c32.slice/crio-2332a360e15e56797fd48d81513b0e83eb08200f3762427b8dcdc42864bc1c76 WatchSource:0}: Error finding container 2332a360e15e56797fd48d81513b0e83eb08200f3762427b8dcdc42864bc1c76: Status 404 returned error can't find the container with id 2332a360e15e56797fd48d81513b0e83eb08200f3762427b8dcdc42864bc1c76
Sep 09 12:38:56.079899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:56.079860    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f" event={"ID":"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32","Type":"ContainerStarted","Data":"9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e"}
Sep 09 12:38:56.079899 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:56.079906    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f" event={"ID":"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32","Type":"ContainerStarted","Data":"2332a360e15e56797fd48d81513b0e83eb08200f3762427b8dcdc42864bc1c76"}
Sep 09 12:38:56.100832 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:56.100779    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f" podStartSLOduration=1.100758115 podStartE2EDuration="1.100758115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:56.098831045 +0000 UTC m=+1680.740450251" watchObservedRunningTime="2026-09-09 12:38:56.100758115 +0000 UTC m=+1680.742377320"
Sep 09 12:38:57.705037 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:57.705002    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"]
Sep 09 12:38:57.710136 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:57.710113    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"
Sep 09 12:38:57.715191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:57.715159    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:38:57.715324 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:57.715230    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:38:57.715829 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:57.715799    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"]
Sep 09 12:38:57.847145 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:57.847107    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da1fd14e-a85f-4e02-80e2-61271f165104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x\" (UID: \"da1fd14e-a85f-4e02-80e2-61271f165104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"
Sep 09 12:38:57.948773 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:57.948737    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da1fd14e-a85f-4e02-80e2-61271f165104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x\" (UID: \"da1fd14e-a85f-4e02-80e2-61271f165104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"
Sep 09 12:38:57.949513 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:57.949487    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da1fd14e-a85f-4e02-80e2-61271f165104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x\" (UID: \"da1fd14e-a85f-4e02-80e2-61271f165104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"
Sep 09 12:38:58.028294 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:58.028253    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"
Sep 09 12:38:58.305573 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:58.305547    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"]
Sep 09 12:38:58.307814 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:38:58.307784    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1fd14e_a85f_4e02_80e2_61271f165104.slice/crio-978dcb9fb69985bff0b0fc001786a4c87b259409184063d7afb984d2fa736c76 WatchSource:0}: Error finding container 978dcb9fb69985bff0b0fc001786a4c87b259409184063d7afb984d2fa736c76: Status 404 returned error can't find the container with id 978dcb9fb69985bff0b0fc001786a4c87b259409184063d7afb984d2fa736c76
Sep 09 12:38:58.983535 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:58.983435    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"]
Sep 09 12:38:58.987783 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:58.987754    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4ec60c-4t2p9"]
Sep 09 12:38:59.102847 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:59.102670    2585 generic.go:356] "Generic (PLEG): container finished" podID="da1fd14e-a85f-4e02-80e2-61271f165104" containerID="a8504da3b674fa095952424d626c20a155c1d68616ded92580ee556b96d04408" exitCode=0
Sep 09 12:38:59.103037 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:59.102937    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x" event={"ID":"da1fd14e-a85f-4e02-80e2-61271f165104","Type":"ContainerDied","Data":"a8504da3b674fa095952424d626c20a155c1d68616ded92580ee556b96d04408"}
Sep 09 12:38:59.103037 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:59.102994    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x" event={"ID":"da1fd14e-a85f-4e02-80e2-61271f165104","Type":"ContainerStarted","Data":"978dcb9fb69985bff0b0fc001786a4c87b259409184063d7afb984d2fa736c76"}
Sep 09 12:38:59.900786 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:38:59.900746    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de475046-e0b3-47dd-81d9-e7c5e455de80" path="/var/lib/kubelet/pods/de475046-e0b3-47dd-81d9-e7c5e455de80/volumes"
Sep 09 12:39:00.243222 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:00.243195    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"
Sep 09 12:39:00.373723 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:00.373665    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da1fd14e-a85f-4e02-80e2-61271f165104-maas-api-service-ca\") pod \"da1fd14e-a85f-4e02-80e2-61271f165104\" (UID: \"da1fd14e-a85f-4e02-80e2-61271f165104\") "
Sep 09 12:39:00.374079 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:00.373970    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da1fd14e-a85f-4e02-80e2-61271f165104-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da1fd14e-a85f-4e02-80e2-61271f165104" (UID: "da1fd14e-a85f-4e02-80e2-61271f165104"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:39:00.475488 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:00.475403    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da1fd14e-a85f-4e02-80e2-61271f165104-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:01.115938 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:01.115907    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"
Sep 09 12:39:01.116135 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:01.115949    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x" event={"ID":"da1fd14e-a85f-4e02-80e2-61271f165104","Type":"ContainerDied","Data":"978dcb9fb69985bff0b0fc001786a4c87b259409184063d7afb984d2fa736c76"}
Sep 09 12:39:01.116135 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:01.116021    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="978dcb9fb69985bff0b0fc001786a4c87b259409184063d7afb984d2fa736c76"
Sep 09 12:39:02.576557 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:02.576520    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"]
Sep 09 12:39:02.601279 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:02.601229    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"]
Sep 09 12:39:02.610374 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:02.610327    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"]
Sep 09 12:39:05.624804 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:05.624765    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:39:05.625223 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:05.624851    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:39:05.637460 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:05.637432    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:39:06.156315 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:06.156275    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:39:13.296098 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.296060    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"]
Sep 09 12:39:13.296547 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.296305    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625" podUID="baceb9d6-9b27-4a4a-be78-1f4730f15109" containerName="payload-pre-processing" containerID="cri-o://df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959" gracePeriod=30
Sep 09 12:39:13.483534 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.483493    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"]
Sep 09 12:39:13.487054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.487024    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-bc0270-dl66x"]
Sep 09 12:39:13.569800 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.569773    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:39:13.608570 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.608405    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx26s\" (UniqueName: \"kubernetes.io/projected/baceb9d6-9b27-4a4a-be78-1f4730f15109-kube-api-access-dx26s\") pod \"baceb9d6-9b27-4a4a-be78-1f4730f15109\" (UID: \"baceb9d6-9b27-4a4a-be78-1f4730f15109\") "
Sep 09 12:39:13.608570 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.608491    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baceb9d6-9b27-4a4a-be78-1f4730f15109-plugins-config-volume\") pod \"baceb9d6-9b27-4a4a-be78-1f4730f15109\" (UID: \"baceb9d6-9b27-4a4a-be78-1f4730f15109\") "
Sep 09 12:39:13.609214 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.609176    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baceb9d6-9b27-4a4a-be78-1f4730f15109-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "baceb9d6-9b27-4a4a-be78-1f4730f15109" (UID: "baceb9d6-9b27-4a4a-be78-1f4730f15109"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:39:13.611898 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.611784    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baceb9d6-9b27-4a4a-be78-1f4730f15109-kube-api-access-dx26s" (OuterVolumeSpecName: "kube-api-access-dx26s") pod "baceb9d6-9b27-4a4a-be78-1f4730f15109" (UID: "baceb9d6-9b27-4a4a-be78-1f4730f15109"). InnerVolumeSpecName "kube-api-access-dx26s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:39:13.710320 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.710265    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx26s\" (UniqueName: \"kubernetes.io/projected/baceb9d6-9b27-4a4a-be78-1f4730f15109-kube-api-access-dx26s\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:13.710320 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.710311    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baceb9d6-9b27-4a4a-be78-1f4730f15109-plugins-config-volume\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:13.900163 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:13.900067    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da1fd14e-a85f-4e02-80e2-61271f165104" path="/var/lib/kubelet/pods/da1fd14e-a85f-4e02-80e2-61271f165104/volumes"
Sep 09 12:39:14.195052 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:14.194964    2585 generic.go:356] "Generic (PLEG): container finished" podID="baceb9d6-9b27-4a4a-be78-1f4730f15109" containerID="df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959" exitCode=0
Sep 09 12:39:14.195052 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:14.195019    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"
Sep 09 12:39:14.195052 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:14.195036    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625" event={"ID":"baceb9d6-9b27-4a4a-be78-1f4730f15109","Type":"ContainerDied","Data":"df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959"}
Sep 09 12:39:14.195283 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:14.195086    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625" event={"ID":"baceb9d6-9b27-4a4a-be78-1f4730f15109","Type":"ContainerDied","Data":"1cf940dd2d1fff888394851cf8bc3ba645cccdc9fd99959c47fdf3f426d524b4"}
Sep 09 12:39:14.195283 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:14.195110    2585 scope.go:117] "RemoveContainer" containerID="df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959"
Sep 09 12:39:14.204778 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:14.204759    2585 scope.go:117] "RemoveContainer" containerID="df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959"
Sep 09 12:39:14.205074 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:39:14.205056    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959\": container with ID starting with df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959 not found: ID does not exist" containerID="df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959"
Sep 09 12:39:14.205124 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:14.205083    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959"} err="failed to get container status \"df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959\": rpc error: code = NotFound desc = could not find container \"df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959\": container with ID starting with df48f1b96be6fda006cccd8ea77b5a0e35394b5cc7fb183a4e4c327d103fe959 not found: ID does not exist"
Sep 09 12:39:14.213834 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:14.213808    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"]
Sep 09 12:39:14.218606 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:14.218579    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-bc0270-78cb846458-lf625"]
Sep 09 12:39:15.899661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:15.899628    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baceb9d6-9b27-4a4a-be78-1f4730f15109" path="/var/lib/kubelet/pods/baceb9d6-9b27-4a4a-be78-1f4730f15109/volumes"
Sep 09 12:39:17.598139 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.598045    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj" podUID="bcc4611a-3bdf-4961-864f-76b26d745c64" containerName="main" containerID="cri-o://72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422" gracePeriod=45
Sep 09 12:39:17.626191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.626141    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj" podUID="628822ce-5179-434a-9479-a307bb2c91c9" containerName="main" containerID="cri-o://6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17" gracePeriod=45
Sep 09 12:39:17.631786 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.631754    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx" podUID="0de13caa-9563-4faa-bd6f-6784747f7813" containerName="main" containerID="cri-o://c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409" gracePeriod=45
Sep 09 12:39:17.894647 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.894613    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:39:17.943440 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.943416    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:39:17.946555 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.946533    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:39:17.949897 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.949876    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-home\") pod \"bcc4611a-3bdf-4961-864f-76b26d745c64\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") "
Sep 09 12:39:17.950021 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.949914    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcc4611a-3bdf-4961-864f-76b26d745c64-tls-certs\") pod \"bcc4611a-3bdf-4961-864f-76b26d745c64\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") "
Sep 09 12:39:17.950021 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.949940    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxrcp\" (UniqueName: \"kubernetes.io/projected/bcc4611a-3bdf-4961-864f-76b26d745c64-kube-api-access-kxrcp\") pod \"bcc4611a-3bdf-4961-864f-76b26d745c64\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") "
Sep 09 12:39:17.950021 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.949995    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-tmp-dir\") pod \"bcc4611a-3bdf-4961-864f-76b26d745c64\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") "
Sep 09 12:39:17.950185 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.950092    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-model-cache\") pod \"bcc4611a-3bdf-4961-864f-76b26d745c64\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") "
Sep 09 12:39:17.950185 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.950126    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-dshm\") pod \"bcc4611a-3bdf-4961-864f-76b26d745c64\" (UID: \"bcc4611a-3bdf-4961-864f-76b26d745c64\") "
Sep 09 12:39:17.950185 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.950127    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-home" (OuterVolumeSpecName: "home") pod "bcc4611a-3bdf-4961-864f-76b26d745c64" (UID: "bcc4611a-3bdf-4961-864f-76b26d745c64"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:17.950397 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.950368    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bcc4611a-3bdf-4961-864f-76b26d745c64" (UID: "bcc4611a-3bdf-4961-864f-76b26d745c64"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:17.950579 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.950558    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:17.950655 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.950584    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:17.950896 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.950806    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-model-cache" (OuterVolumeSpecName: "model-cache") pod "bcc4611a-3bdf-4961-864f-76b26d745c64" (UID: "bcc4611a-3bdf-4961-864f-76b26d745c64"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:17.952343 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.952316    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcc4611a-3bdf-4961-864f-76b26d745c64-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bcc4611a-3bdf-4961-864f-76b26d745c64" (UID: "bcc4611a-3bdf-4961-864f-76b26d745c64"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:39:17.952524 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.952502    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-dshm" (OuterVolumeSpecName: "dshm") pod "bcc4611a-3bdf-4961-864f-76b26d745c64" (UID: "bcc4611a-3bdf-4961-864f-76b26d745c64"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:17.953398 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:17.953380    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcc4611a-3bdf-4961-864f-76b26d745c64-kube-api-access-kxrcp" (OuterVolumeSpecName: "kube-api-access-kxrcp") pod "bcc4611a-3bdf-4961-864f-76b26d745c64" (UID: "bcc4611a-3bdf-4961-864f-76b26d745c64"). InnerVolumeSpecName "kube-api-access-kxrcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:39:18.051206 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051175    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-tmp-dir\") pod \"0de13caa-9563-4faa-bd6f-6784747f7813\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") "
Sep 09 12:39:18.051206 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051211    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0de13caa-9563-4faa-bd6f-6784747f7813-tls-certs\") pod \"0de13caa-9563-4faa-bd6f-6784747f7813\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") "
Sep 09 12:39:18.051475 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051246    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8gxk\" (UniqueName: \"kubernetes.io/projected/0de13caa-9563-4faa-bd6f-6784747f7813-kube-api-access-f8gxk\") pod \"0de13caa-9563-4faa-bd6f-6784747f7813\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") "
Sep 09 12:39:18.051475 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051271    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-model-cache\") pod \"628822ce-5179-434a-9479-a307bb2c91c9\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") "
Sep 09 12:39:18.051475 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051453    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-home\") pod \"0de13caa-9563-4faa-bd6f-6784747f7813\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") "
Sep 09 12:39:18.051639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051490    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-dshm\") pod \"0de13caa-9563-4faa-bd6f-6784747f7813\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") "
Sep 09 12:39:18.051639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051510    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "628822ce-5179-434a-9479-a307bb2c91c9" (UID: "628822ce-5179-434a-9479-a307bb2c91c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:18.051639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051543    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf8hb\" (UniqueName: \"kubernetes.io/projected/628822ce-5179-434a-9479-a307bb2c91c9-kube-api-access-mf8hb\") pod \"628822ce-5179-434a-9479-a307bb2c91c9\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") "
Sep 09 12:39:18.051639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051510    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0de13caa-9563-4faa-bd6f-6784747f7813" (UID: "0de13caa-9563-4faa-bd6f-6784747f7813"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:18.051639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051594    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-tmp-dir\") pod \"628822ce-5179-434a-9479-a307bb2c91c9\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") "
Sep 09 12:39:18.051639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051630    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-model-cache\") pod \"0de13caa-9563-4faa-bd6f-6784747f7813\" (UID: \"0de13caa-9563-4faa-bd6f-6784747f7813\") "
Sep 09 12:39:18.051999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051653    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-home\") pod \"628822ce-5179-434a-9479-a307bb2c91c9\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") "
Sep 09 12:39:18.051999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051690    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-home" (OuterVolumeSpecName: "home") pod "0de13caa-9563-4faa-bd6f-6784747f7813" (UID: "0de13caa-9563-4faa-bd6f-6784747f7813"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:18.051999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051765    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628822ce-5179-434a-9479-a307bb2c91c9-tls-certs\") pod \"628822ce-5179-434a-9479-a307bb2c91c9\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") "
Sep 09 12:39:18.051999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051811    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-dshm\") pod \"628822ce-5179-434a-9479-a307bb2c91c9\" (UID: \"628822ce-5179-434a-9479-a307bb2c91c9\") "
Sep 09 12:39:18.051999 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.051897    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "628822ce-5179-434a-9479-a307bb2c91c9" (UID: "628822ce-5179-434a-9479-a307bb2c91c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:18.052286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052075    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-model-cache" (OuterVolumeSpecName: "model-cache") pod "0de13caa-9563-4faa-bd6f-6784747f7813" (UID: "0de13caa-9563-4faa-bd6f-6784747f7813"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:18.052286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052198    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.052286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052215    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.052286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052228    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.052286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052209    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-home" (OuterVolumeSpecName: "home") pod "628822ce-5179-434a-9479-a307bb2c91c9" (UID: "628822ce-5179-434a-9479-a307bb2c91c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:18.052286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052242    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcc4611a-3bdf-4961-864f-76b26d745c64-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.052286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052255    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxrcp\" (UniqueName: \"kubernetes.io/projected/bcc4611a-3bdf-4961-864f-76b26d745c64-kube-api-access-kxrcp\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.052286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052270    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.052286 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052282    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.052955 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052303    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.052955 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.052316    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcc4611a-3bdf-4961-864f-76b26d745c64-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.054098 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.054071    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0de13caa-9563-4faa-bd6f-6784747f7813-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0de13caa-9563-4faa-bd6f-6784747f7813" (UID: "0de13caa-9563-4faa-bd6f-6784747f7813"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:39:18.054238 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.054196    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0de13caa-9563-4faa-bd6f-6784747f7813-kube-api-access-f8gxk" (OuterVolumeSpecName: "kube-api-access-f8gxk") pod "0de13caa-9563-4faa-bd6f-6784747f7813" (UID: "0de13caa-9563-4faa-bd6f-6784747f7813"). InnerVolumeSpecName "kube-api-access-f8gxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:39:18.054283 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.054251    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/628822ce-5179-434a-9479-a307bb2c91c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "628822ce-5179-434a-9479-a307bb2c91c9" (UID: "628822ce-5179-434a-9479-a307bb2c91c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:39:18.054490 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.054472    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-dshm" (OuterVolumeSpecName: "dshm") pod "0de13caa-9563-4faa-bd6f-6784747f7813" (UID: "0de13caa-9563-4faa-bd6f-6784747f7813"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:18.054746 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.054728    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/628822ce-5179-434a-9479-a307bb2c91c9-kube-api-access-mf8hb" (OuterVolumeSpecName: "kube-api-access-mf8hb") pod "628822ce-5179-434a-9479-a307bb2c91c9" (UID: "628822ce-5179-434a-9479-a307bb2c91c9"). InnerVolumeSpecName "kube-api-access-mf8hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:39:18.054746 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.054736    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-dshm" (OuterVolumeSpecName: "dshm") pod "628822ce-5179-434a-9479-a307bb2c91c9" (UID: "628822ce-5179-434a-9479-a307bb2c91c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:39:18.153331 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.153245    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628822ce-5179-434a-9479-a307bb2c91c9-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.153331 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.153274    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.153331 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.153282    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0de13caa-9563-4faa-bd6f-6784747f7813-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.153331 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.153291    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8gxk\" (UniqueName: \"kubernetes.io/projected/0de13caa-9563-4faa-bd6f-6784747f7813-kube-api-access-f8gxk\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.153331 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.153299    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0de13caa-9563-4faa-bd6f-6784747f7813-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.153331 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.153308    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf8hb\" (UniqueName: \"kubernetes.io/projected/628822ce-5179-434a-9479-a307bb2c91c9-kube-api-access-mf8hb\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.153331 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.153317    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628822ce-5179-434a-9479-a307bb2c91c9-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:39:18.221888 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.221854    2585 generic.go:356] "Generic (PLEG): container finished" podID="bcc4611a-3bdf-4961-864f-76b26d745c64" containerID="72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422" exitCode=0
Sep 09 12:39:18.222093 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.221913    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"
Sep 09 12:39:18.222093 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.221951    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj" event={"ID":"bcc4611a-3bdf-4961-864f-76b26d745c64","Type":"ContainerDied","Data":"72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422"}
Sep 09 12:39:18.222093 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.222000    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj" event={"ID":"bcc4611a-3bdf-4961-864f-76b26d745c64","Type":"ContainerDied","Data":"e5d67b13ff3cc011949453a722cd2c7f14fb33618bc825fe7ad82b2c30f47d51"}
Sep 09 12:39:18.222093 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.222028    2585 scope.go:117] "RemoveContainer" containerID="72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422"
Sep 09 12:39:18.223526 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.223503    2585 generic.go:356] "Generic (PLEG): container finished" podID="0de13caa-9563-4faa-bd6f-6784747f7813" containerID="c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409" exitCode=0
Sep 09 12:39:18.223637 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.223560    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"
Sep 09 12:39:18.223637 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.223582    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx" event={"ID":"0de13caa-9563-4faa-bd6f-6784747f7813","Type":"ContainerDied","Data":"c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409"}
Sep 09 12:39:18.223637 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.223627    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx" event={"ID":"0de13caa-9563-4faa-bd6f-6784747f7813","Type":"ContainerDied","Data":"15b0884b14b27755c450d43b5b77f131e110c0330b5cf4e3b2a0dcd7767b99b7"}
Sep 09 12:39:18.225096 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.225078    2585 generic.go:356] "Generic (PLEG): container finished" podID="628822ce-5179-434a-9479-a307bb2c91c9" containerID="6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17" exitCode=0
Sep 09 12:39:18.225199 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.225114    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"
Sep 09 12:39:18.225199 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.225136    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj" event={"ID":"628822ce-5179-434a-9479-a307bb2c91c9","Type":"ContainerDied","Data":"6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17"}
Sep 09 12:39:18.225199 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.225164    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj" event={"ID":"628822ce-5179-434a-9479-a307bb2c91c9","Type":"ContainerDied","Data":"e7c1f4e05df88aa6ddb7f8a61d6c53dc75ebbc24664b7edee380f535f76eae23"}
Sep 09 12:39:18.233764 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.233743    2585 scope.go:117] "RemoveContainer" containerID="72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422"
Sep 09 12:39:18.234082 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:39:18.234058    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422\": container with ID starting with 72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422 not found: ID does not exist" containerID="72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422"
Sep 09 12:39:18.234191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.234087    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422"} err="failed to get container status \"72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422\": rpc error: code = NotFound desc = could not find container \"72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422\": container with ID starting with 72c842091f98ba0bb5d844f97ce42d8306c26756db47c787fb4af6370f764422 not found: ID does not exist"
Sep 09 12:39:18.234191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.234101    2585 scope.go:117] "RemoveContainer" containerID="c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409"
Sep 09 12:39:18.243085 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.243066    2585 scope.go:117] "RemoveContainer" containerID="c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409"
Sep 09 12:39:18.243344 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:39:18.243326    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409\": container with ID starting with c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409 not found: ID does not exist" containerID="c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409"
Sep 09 12:39:18.243390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.243352    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409"} err="failed to get container status \"c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409\": rpc error: code = NotFound desc = could not find container \"c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409\": container with ID starting with c090c819159999f5d7ef300dcbf5df2cefd56cf6c44c1a11faf28ae591db2409 not found: ID does not exist"
Sep 09 12:39:18.243390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.243367    2585 scope.go:117] "RemoveContainer" containerID="6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17"
Sep 09 12:39:18.246760 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.246737    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"]
Sep 09 12:39:18.253935 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.252165    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/distinct-2-w4-bc0270-kserve-55f5445d4b-wbvmj"]
Sep 09 12:39:18.254715 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.254672    2585 scope.go:117] "RemoveContainer" containerID="6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17"
Sep 09 12:39:18.255022 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:39:18.255003    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17\": container with ID starting with 6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17 not found: ID does not exist" containerID="6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17"
Sep 09 12:39:18.255092 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.255030    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17"} err="failed to get container status \"6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17\": rpc error: code = NotFound desc = could not find container \"6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17\": container with ID starting with 6ef865a6ad249988e59bde1c94ab2f8a31b23c47aeee59690192e7c9e5207d17 not found: ID does not exist"
Sep 09 12:39:18.265287 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.265265    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"]
Sep 09 12:39:18.269239 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.269218    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/unconfigured-w4-bc0270-kserve-5965bb698b-tm4zx"]
Sep 09 12:39:18.279748 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.279728    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"]
Sep 09 12:39:18.284200 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:18.284181    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-bc0270/simulator-w4-bc0270-kserve-7b66b78459-hgbwj"]
Sep 09 12:39:19.899657 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:19.899621    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0de13caa-9563-4faa-bd6f-6784747f7813" path="/var/lib/kubelet/pods/0de13caa-9563-4faa-bd6f-6784747f7813/volumes"
Sep 09 12:39:19.900079 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:19.900063    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="628822ce-5179-434a-9479-a307bb2c91c9" path="/var/lib/kubelet/pods/628822ce-5179-434a-9479-a307bb2c91c9/volumes"
Sep 09 12:39:19.900423 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:39:19.900409    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcc4611a-3bdf-4961-864f-76b26d745c64" path="/var/lib/kubelet/pods/bcc4611a-3bdf-4961-864f-76b26d745c64/volumes"
Sep 09 12:40:28.718332 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:28.718299    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"]
Sep 09 12:40:43.738922 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:43.738877    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f" podUID="1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" containerName="main" containerID="cri-o://9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e" gracePeriod=45
Sep 09 12:40:44.002982 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.002908    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:40:44.117173 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117141    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tls-certs\") pod \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") "
Sep 09 12:40:44.117340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117180    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g86l4\" (UniqueName: \"kubernetes.io/projected/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-kube-api-access-g86l4\") pod \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") "
Sep 09 12:40:44.117340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117200    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-dshm\") pod \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") "
Sep 09 12:40:44.117340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117250    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-home\") pod \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") "
Sep 09 12:40:44.117340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117269    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tmp-dir\") pod \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") "
Sep 09 12:40:44.117340 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117306    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-model-cache\") pod \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\" (UID: \"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32\") "
Sep 09 12:40:44.117593 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117508    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-home" (OuterVolumeSpecName: "home") pod "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" (UID: "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:40:44.117593 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117531    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" (UID: "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:40:44.117723 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117620    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-model-cache" (OuterVolumeSpecName: "model-cache") pod "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" (UID: "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:40:44.117921 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117896    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:40:44.118029 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117925    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:40:44.118029 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.117940    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:40:44.119476 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.119454    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-dshm" (OuterVolumeSpecName: "dshm") pod "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" (UID: "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:40:44.119566 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.119476    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-kube-api-access-g86l4" (OuterVolumeSpecName: "kube-api-access-g86l4") pod "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" (UID: "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32"). InnerVolumeSpecName "kube-api-access-g86l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:40:44.119676 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.119663    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" (UID: "1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:40:44.219069 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.219030    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:40:44.219069 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.219062    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g86l4\" (UniqueName: \"kubernetes.io/projected/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-kube-api-access-g86l4\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:40:44.219069 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.219074    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:40:44.741365 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.741328    2585 generic.go:356] "Generic (PLEG): container finished" podID="1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" containerID="9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e" exitCode=0
Sep 09 12:40:44.741854 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.741377    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"
Sep 09 12:40:44.741854 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.741397    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f" event={"ID":"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32","Type":"ContainerDied","Data":"9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e"}
Sep 09 12:40:44.741854 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.741428    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f" event={"ID":"1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32","Type":"ContainerDied","Data":"2332a360e15e56797fd48d81513b0e83eb08200f3762427b8dcdc42864bc1c76"}
Sep 09 12:40:44.741854 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.741444    2585 scope.go:117] "RemoveContainer" containerID="9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e"
Sep 09 12:40:44.754613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.754554    2585 scope.go:117] "RemoveContainer" containerID="9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e"
Sep 09 12:40:44.755014 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:40:44.754992    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e\": container with ID starting with 9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e not found: ID does not exist" containerID="9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e"
Sep 09 12:40:44.755093 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.755025    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e"} err="failed to get container status \"9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e\": rpc error: code = NotFound desc = could not find container \"9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e\": container with ID starting with 9c29c6b06e8a17ea46d4671ae8c0f80f86e75ae6bf4376007ab5f85d228a096e not found: ID does not exist"
Sep 09 12:40:44.764951 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.764924    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"]
Sep 09 12:40:44.773949 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:44.771570    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-679f45/sub-test-model-679f45-kserve-79f66695f7-zvt6f"]
Sep 09 12:40:45.899769 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:45.899733    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" path="/var/lib/kubelet/pods/1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32/volumes"
Sep 09 12:40:56.248797 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:40:56.248764    2585 scope.go:117] "RemoveContainer" containerID="9cb1468fa0f3300833d8da16f2e5ab3443cb8f7d39a6a4f129e4d0a55f3fdc27"
Sep 09 12:42:15.120745 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.120690    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"]
Sep 09 12:42:15.121383 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121361    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baceb9d6-9b27-4a4a-be78-1f4730f15109" containerName="payload-pre-processing"
Sep 09 12:42:15.121429 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121389    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="baceb9d6-9b27-4a4a-be78-1f4730f15109" containerName="payload-pre-processing"
Sep 09 12:42:15.121429 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121402    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="628822ce-5179-434a-9479-a307bb2c91c9" containerName="main"
Sep 09 12:42:15.121429 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121411    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="628822ce-5179-434a-9479-a307bb2c91c9" containerName="main"
Sep 09 12:42:15.121523 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121434    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0de13caa-9563-4faa-bd6f-6784747f7813" containerName="main"
Sep 09 12:42:15.121523 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121443    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0de13caa-9563-4faa-bd6f-6784747f7813" containerName="main"
Sep 09 12:42:15.121523 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121466    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" containerName="main"
Sep 09 12:42:15.121523 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121474    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" containerName="main"
Sep 09 12:42:15.121523 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121493    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcc4611a-3bdf-4961-864f-76b26d745c64" containerName="main"
Sep 09 12:42:15.121523 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121502    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcc4611a-3bdf-4961-864f-76b26d745c64" containerName="main"
Sep 09 12:42:15.121523 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121516    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da1fd14e-a85f-4e02-80e2-61271f165104" containerName="revoke-keys"
Sep 09 12:42:15.121762 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121524    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="da1fd14e-a85f-4e02-80e2-61271f165104" containerName="revoke-keys"
Sep 09 12:42:15.121762 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121625    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="baceb9d6-9b27-4a4a-be78-1f4730f15109" containerName="payload-pre-processing"
Sep 09 12:42:15.121762 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121638    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8ebf21-e94d-46e8-bb5f-8b82b51b2c32" containerName="main"
Sep 09 12:42:15.121762 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121647    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="da1fd14e-a85f-4e02-80e2-61271f165104" containerName="revoke-keys"
Sep 09 12:42:15.121762 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121658    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcc4611a-3bdf-4961-864f-76b26d745c64" containerName="main"
Sep 09 12:42:15.121762 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121670    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="628822ce-5179-434a-9479-a307bb2c91c9" containerName="main"
Sep 09 12:42:15.121762 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.121679    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0de13caa-9563-4faa-bd6f-6784747f7813" containerName="main"
Sep 09 12:42:15.125346 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.125325    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.127964 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.127939    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-8ec58b-openshift-default-dockercfg-m57wp\""
Sep 09 12:42:15.137380 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.137357    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"]
Sep 09 12:42:15.227282 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.227235    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqcp7\" (UniqueName: \"kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-kube-api-access-xqcp7\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.227612 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.227590    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92f716ca-a856-48d6-b97d-e99807e70a82-istio-podinfo\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.227792 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.227774    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-istio-token\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.227932 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.227917    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-socket\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.228088 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.228074    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-credential-socket\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.228194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.228180    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-certs\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.228314 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.228301    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-data\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.228448 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.228434    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92f716ca-a856-48d6-b97d-e99807e70a82-istiod-ca-cert\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.228560 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.228545    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-envoy\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329304 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329266    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92f716ca-a856-48d6-b97d-e99807e70a82-istio-podinfo\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329312    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-istio-token\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329341    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-socket\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329381    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-credential-socket\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329397    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-certs\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329417    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-data\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329493 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329457    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92f716ca-a856-48d6-b97d-e99807e70a82-istiod-ca-cert\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329872 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329499    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-envoy\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329872 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329532    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqcp7\" (UniqueName: \"kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-kube-api-access-xqcp7\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329872 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329862    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-socket\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.329975 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.329876    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-credential-socket\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.330083 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.330052    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-certs\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.330229 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.330207    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-data\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.330405 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.330383    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92f716ca-a856-48d6-b97d-e99807e70a82-istiod-ca-cert\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.332005 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.331988    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92f716ca-a856-48d6-b97d-e99807e70a82-istio-podinfo\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.332303 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.332286    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-envoy\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.339377 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.339346    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-istio-token\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.339566 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.339546    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqcp7\" (UniqueName: \"kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-kube-api-access-xqcp7\") pod \"e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") " pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.437757 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.437631    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:15.717450 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.717422    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"]
Sep 09 12:42:15.718974 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:42:15.718935    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92f716ca_a856_48d6_b97d_e99807e70a82.slice/crio-25644945ee02cae8348b5e2c2b4089d5bd8b732e1777cfacb25e026075e82ab4 WatchSource:0}: Error finding container 25644945ee02cae8348b5e2c2b4089d5bd8b732e1777cfacb25e026075e82ab4: Status 404 returned error can't find the container with id 25644945ee02cae8348b5e2c2b4089d5bd8b732e1777cfacb25e026075e82ab4
Sep 09 12:42:15.721392 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.721350    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:42:15.721481 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.721443    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:42:15.721519 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:15.721489    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:42:16.261245 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:16.261206    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" event={"ID":"92f716ca-a856-48d6-b97d-e99807e70a82","Type":"ContainerStarted","Data":"8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0"}
Sep 09 12:42:16.261639 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:16.261263    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" event={"ID":"92f716ca-a856-48d6-b97d-e99807e70a82","Type":"ContainerStarted","Data":"25644945ee02cae8348b5e2c2b4089d5bd8b732e1777cfacb25e026075e82ab4"}
Sep 09 12:42:16.287609 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:16.287553    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" podStartSLOduration=1.287535576 podStartE2EDuration="1.287535576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:42:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:42:16.284743421 +0000 UTC m=+1880.926362630" watchObservedRunningTime="2026-09-09 12:42:16.287535576 +0000 UTC m=+1880.929154780"
Sep 09 12:42:16.438409 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:16.438374    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:16.443491 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:16.443465    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:17.267540 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:17.267500    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:17.268518 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:17.268498    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:48.309931 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:48.309884    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 12:42:48.310376 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:48.309965    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:42:48.610390 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:48.610314    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-chp7l_7b6e2ff7-9e23-49c1-aa14-b29047aeef39/payload-processing/0.log"
Sep 09 12:42:50.461894 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:50.461858    2585 generic.go:356] "Generic (PLEG): container finished" podID="92f716ca-a856-48d6-b97d-e99807e70a82" containerID="8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0" exitCode=0
Sep 09 12:42:50.462304 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:50.461908    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" event={"ID":"92f716ca-a856-48d6-b97d-e99807e70a82","Type":"ContainerDied","Data":"8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0"}
Sep 09 12:42:50.462351 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:50.462318    2585 scope.go:117] "RemoveContainer" containerID="8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0"
Sep 09 12:42:50.463519 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:50.463492    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:42:50.463589 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:50.463561    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:42:50.463626 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:50.463590    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:42:51.476152 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:51.476112    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" event={"ID":"92f716ca-a856-48d6-b97d-e99807e70a82","Type":"ContainerStarted","Data":"abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a"}
Sep 09 12:42:52.438709 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.438665    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:52.438973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.438950    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 09 12:42:52.439020 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.438998    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 09 12:42:52.700544 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.700448    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"]
Sep 09 12:42:52.704779 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.704759    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.707348 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.707323    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-232ce2\"/\"openshift-service-ca.crt\""
Sep 09 12:42:52.708362 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.708340    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-232ce2\"/\"default-dockercfg-gkqdb\""
Sep 09 12:42:52.708481 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.708394    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-232ce2\"/\"ipp-route-232ce2-kserve-self-signed-certs\""
Sep 09 12:42:52.708481 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.708431    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-232ce2\"/\"kube-root-ca.crt\""
Sep 09 12:42:52.716528 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.716503    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"]
Sep 09 12:42:52.834169 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.834133    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tls-certs\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.834385 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.834238    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-dshm\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.834385 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.834293    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqbvs\" (UniqueName: \"kubernetes.io/projected/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-kube-api-access-vqbvs\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.834385 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.834323    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-model-cache\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.834516 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.834395    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tmp-dir\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.834516 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.834478    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-home\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.935453 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.935413    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-dshm\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.935659 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.935471    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqbvs\" (UniqueName: \"kubernetes.io/projected/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-kube-api-access-vqbvs\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.935659 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.935492    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-model-cache\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.935659 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.935521    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tmp-dir\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.935659 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.935578    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-home\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.935659 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.935606    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tls-certs\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.935963 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.935942    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tmp-dir\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.936018 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.935995    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-model-cache\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.936068 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.936034    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-home\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.938014 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.937991    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-dshm\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.938259 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.938245    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tls-certs\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:52.945564 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:52.945538    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqbvs\" (UniqueName: \"kubernetes.io/projected/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-kube-api-access-vqbvs\") pod \"ipp-route-232ce2-kserve-76d8bb986c-fr2bg\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") " pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:53.014287 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:53.014258    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:42:53.317644 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:53.317615    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"]
Sep 09 12:42:53.319225 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:42:53.319192    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba5a583a_3325_4577_8ebc_7a5ce0f261fb.slice/crio-0ca8d1e68ede5b29e0fbf30cda0f78f3223b2bba81007cf93d2c4a95cad57368 WatchSource:0}: Error finding container 0ca8d1e68ede5b29e0fbf30cda0f78f3223b2bba81007cf93d2c4a95cad57368: Status 404 returned error can't find the container with id 0ca8d1e68ede5b29e0fbf30cda0f78f3223b2bba81007cf93d2c4a95cad57368
Sep 09 12:42:53.438642 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:53.438603    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 09 12:42:53.438837 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:53.438673    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 09 12:42:53.489328 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:53.489297    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg" event={"ID":"ba5a583a-3325-4577-8ebc-7a5ce0f261fb","Type":"ContainerStarted","Data":"a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc"}
Sep 09 12:42:53.489770 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:53.489346    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg" event={"ID":"ba5a583a-3325-4577-8ebc-7a5ce0f261fb","Type":"ContainerStarted","Data":"0ca8d1e68ede5b29e0fbf30cda0f78f3223b2bba81007cf93d2c4a95cad57368"}
Sep 09 12:42:53.510540 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:53.510484    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg" podStartSLOduration=1.510465731 podStartE2EDuration="1.510465731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:42:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:42:53.507639107 +0000 UTC m=+1918.149258312" watchObservedRunningTime="2026-09-09 12:42:53.510465731 +0000 UTC m=+1918.152084934"
Sep 09 12:42:54.438831 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:54.438792    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 09 12:42:54.439235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:54.438852    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 09 12:42:55.438100 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:55.438064    2585 patch_prober.go:28] interesting pod/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 09 12:42:55.438296 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:55.438121    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 09 12:42:56.441895 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:56.441857    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:56.442345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:56.442154    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:42:56.442754 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:42:56.442738    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:43:03.014425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:03.014384    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:43:03.014973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:03.014438    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:43:03.028028 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:03.028001    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:43:03.556502 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:03.556465    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:43:19.764324 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:19.764289    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-chp7l_7b6e2ff7-9e23-49c1-aa14-b29047aeef39/payload-processing/0.log"
Sep 09 12:43:49.358049 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:49.358014    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"]
Sep 09 12:43:49.361645 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:49.361627    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"
Sep 09 12:43:49.364103 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:49.364084    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:43:49.364169 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:49.364102    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:43:49.369627 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:49.369600    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"]
Sep 09 12:43:49.399202 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:49.399162    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc8dee5-a342-490d-a22f-f90f67b47edd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl\" (UID: \"1fc8dee5-a342-490d-a22f-f90f67b47edd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"
Sep 09 12:43:49.499865 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:49.499829    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc8dee5-a342-490d-a22f-f90f67b47edd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl\" (UID: \"1fc8dee5-a342-490d-a22f-f90f67b47edd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"
Sep 09 12:43:49.500480 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:49.500449    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc8dee5-a342-490d-a22f-f90f67b47edd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl\" (UID: \"1fc8dee5-a342-490d-a22f-f90f67b47edd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"
Sep 09 12:43:49.673828 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:49.673738    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"
Sep 09 12:43:49.949529 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:49.949501    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"]
Sep 09 12:43:49.952016 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:43:49.951985    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc8dee5_a342_490d_a22f_f90f67b47edd.slice/crio-e81146a9cd6d9306b0a16f7cd43aa20356af473bd0237f673c59e1f93882d6e7 WatchSource:0}: Error finding container e81146a9cd6d9306b0a16f7cd43aa20356af473bd0237f673c59e1f93882d6e7: Status 404 returned error can't find the container with id e81146a9cd6d9306b0a16f7cd43aa20356af473bd0237f673c59e1f93882d6e7
Sep 09 12:43:50.808529 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:50.808490    2585 generic.go:356] "Generic (PLEG): container finished" podID="1fc8dee5-a342-490d-a22f-f90f67b47edd" containerID="703036c81fb83a7b27a6747e3475ebf694abf129aa066fb4055a8684b36bf1b7" exitCode=0
Sep 09 12:43:50.809001 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:50.808578    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl" event={"ID":"1fc8dee5-a342-490d-a22f-f90f67b47edd","Type":"ContainerDied","Data":"703036c81fb83a7b27a6747e3475ebf694abf129aa066fb4055a8684b36bf1b7"}
Sep 09 12:43:50.809001 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:50.808665    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl" event={"ID":"1fc8dee5-a342-490d-a22f-f90f67b47edd","Type":"ContainerStarted","Data":"e81146a9cd6d9306b0a16f7cd43aa20356af473bd0237f673c59e1f93882d6e7"}
Sep 09 12:43:51.956088 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:51.956063    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"
Sep 09 12:43:52.023452 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:52.023410    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc8dee5-a342-490d-a22f-f90f67b47edd-maas-api-service-ca\") pod \"1fc8dee5-a342-490d-a22f-f90f67b47edd\" (UID: \"1fc8dee5-a342-490d-a22f-f90f67b47edd\") "
Sep 09 12:43:52.023799 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:52.023776    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fc8dee5-a342-490d-a22f-f90f67b47edd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1fc8dee5-a342-490d-a22f-f90f67b47edd" (UID: "1fc8dee5-a342-490d-a22f-f90f67b47edd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:43:52.124501 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:52.124404    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc8dee5-a342-490d-a22f-f90f67b47edd-maas-api-service-ca\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:43:52.823215 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:52.823173    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl" event={"ID":"1fc8dee5-a342-490d-a22f-f90f67b47edd","Type":"ContainerDied","Data":"e81146a9cd6d9306b0a16f7cd43aa20356af473bd0237f673c59e1f93882d6e7"}
Sep 09 12:43:52.823394 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:52.823234    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e81146a9cd6d9306b0a16f7cd43aa20356af473bd0237f673c59e1f93882d6e7"
Sep 09 12:43:52.823394 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:52.823193    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"
Sep 09 12:43:55.911484 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:55.911447    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"]
Sep 09 12:43:55.917401 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:55.917369    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-330769-68zwl"]
Sep 09 12:43:56.289476 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:56.289440    2585 scope.go:117] "RemoveContainer" containerID="540f486927a5e1b8a0b5016be1b86c788d62ce95db051a3076e8c5d382620285"
Sep 09 12:43:57.899561 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:43:57.899524    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fc8dee5-a342-490d-a22f-f90f67b47edd" path="/var/lib/kubelet/pods/1fc8dee5-a342-490d-a22f-f90f67b47edd/volumes"
Sep 09 12:44:29.510027 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:29.509984    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"]
Sep 09 12:44:44.530365 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.530319    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg" podUID="ba5a583a-3325-4577-8ebc-7a5ce0f261fb" containerName="main" containerID="cri-o://a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc" gracePeriod=45
Sep 09 12:44:44.789623 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.789563    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:44:44.868878 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.868848    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-dshm\") pod \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") "
Sep 09 12:44:44.869063 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.868896    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-model-cache\") pod \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") "
Sep 09 12:44:44.869063 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.868920    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tmp-dir\") pod \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") "
Sep 09 12:44:44.869063 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.868943    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqbvs\" (UniqueName: \"kubernetes.io/projected/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-kube-api-access-vqbvs\") pod \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") "
Sep 09 12:44:44.869063 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.868968    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-home\") pod \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") "
Sep 09 12:44:44.869063 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.868989    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tls-certs\") pod \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\" (UID: \"ba5a583a-3325-4577-8ebc-7a5ce0f261fb\") "
Sep 09 12:44:44.869345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.869170    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba5a583a-3325-4577-8ebc-7a5ce0f261fb" (UID: "ba5a583a-3325-4577-8ebc-7a5ce0f261fb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:44.869345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.869232    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba5a583a-3325-4577-8ebc-7a5ce0f261fb" (UID: "ba5a583a-3325-4577-8ebc-7a5ce0f261fb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:44.869345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.869242    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-home" (OuterVolumeSpecName: "home") pod "ba5a583a-3325-4577-8ebc-7a5ce0f261fb" (UID: "ba5a583a-3325-4577-8ebc-7a5ce0f261fb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:44.869498 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.869444    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:44:44.869498 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.869458    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:44:44.869498 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.869471    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:44:44.871304 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.871284    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-dshm" (OuterVolumeSpecName: "dshm") pod "ba5a583a-3325-4577-8ebc-7a5ce0f261fb" (UID: "ba5a583a-3325-4577-8ebc-7a5ce0f261fb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:44.871391 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.871335    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba5a583a-3325-4577-8ebc-7a5ce0f261fb" (UID: "ba5a583a-3325-4577-8ebc-7a5ce0f261fb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:44:44.871445 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.871379    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-kube-api-access-vqbvs" (OuterVolumeSpecName: "kube-api-access-vqbvs") pod "ba5a583a-3325-4577-8ebc-7a5ce0f261fb" (UID: "ba5a583a-3325-4577-8ebc-7a5ce0f261fb"). InnerVolumeSpecName "kube-api-access-vqbvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:44:44.971031 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.970989    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:44:44.971031 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.971026    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqbvs\" (UniqueName: \"kubernetes.io/projected/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-kube-api-access-vqbvs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:44:44.971031 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:44.971041    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5a583a-3325-4577-8ebc-7a5ce0f261fb-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:44:45.120077 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:45.119992    2585 generic.go:356] "Generic (PLEG): container finished" podID="ba5a583a-3325-4577-8ebc-7a5ce0f261fb" containerID="a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc" exitCode=0
Sep 09 12:44:45.120077 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:45.120037    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"
Sep 09 12:44:45.120276 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:45.120078    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg" event={"ID":"ba5a583a-3325-4577-8ebc-7a5ce0f261fb","Type":"ContainerDied","Data":"a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc"}
Sep 09 12:44:45.120276 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:45.120121    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg" event={"ID":"ba5a583a-3325-4577-8ebc-7a5ce0f261fb","Type":"ContainerDied","Data":"0ca8d1e68ede5b29e0fbf30cda0f78f3223b2bba81007cf93d2c4a95cad57368"}
Sep 09 12:44:45.120276 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:45.120139    2585 scope.go:117] "RemoveContainer" containerID="a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc"
Sep 09 12:44:45.129509 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:45.129492    2585 scope.go:117] "RemoveContainer" containerID="a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc"
Sep 09 12:44:45.129773 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:44:45.129754    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc\": container with ID starting with a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc not found: ID does not exist" containerID="a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc"
Sep 09 12:44:45.129839 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:45.129781    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc"} err="failed to get container status \"a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc\": rpc error: code = NotFound desc = could not find container \"a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc\": container with ID starting with a8e03fc30bcf76397e166d72427320e1d8ee0e26d935f1cd39398e8bc55846bc not found: ID does not exist"
Sep 09 12:44:45.142404 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:45.142386    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"]
Sep 09 12:44:45.146796 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:45.146779    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-232ce2/ipp-route-232ce2-kserve-76d8bb986c-fr2bg"]
Sep 09 12:44:45.901270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:45.901231    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba5a583a-3325-4577-8ebc-7a5ce0f261fb" path="/var/lib/kubelet/pods/ba5a583a-3325-4577-8ebc-7a5ce0f261fb/volumes"
Sep 09 12:44:56.311295 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:56.311257    2585 scope.go:117] "RemoveContainer" containerID="dab7027277d7d16c3429640dd5942396d405894370dd06163071f6cf21a3314b"
Sep 09 12:44:56.319900 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:56.319880    2585 scope.go:117] "RemoveContainer" containerID="c856465bdb19494afdbb04c9426503570248f448fdddb33fc066c36a7ac53661"
Sep 09 12:44:56.327571 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:56.327549    2585 scope.go:117] "RemoveContainer" containerID="b827e6763e1a577b408eaae29f3b30fd4b84ac3813c9acdbfb6c73c6a01b0ded"
Sep 09 12:44:56.334977 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:44:56.334955    2585 scope.go:117] "RemoveContainer" containerID="a02fb4e4cd57f44deb45fb9a0e94c8c5e8173da9ea7b4d650e0ca59db12ed055"
Sep 09 12:45:00.169437 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.169402    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh"]
Sep 09 12:45:00.170002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.169896    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc8dee5-a342-490d-a22f-f90f67b47edd" containerName="revoke-keys"
Sep 09 12:45:00.170002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.169909    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc8dee5-a342-490d-a22f-f90f67b47edd" containerName="revoke-keys"
Sep 09 12:45:00.170002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.169923    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba5a583a-3325-4577-8ebc-7a5ce0f261fb" containerName="main"
Sep 09 12:45:00.170002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.169930    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba5a583a-3325-4577-8ebc-7a5ce0f261fb" containerName="main"
Sep 09 12:45:00.170002 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.170003    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc8dee5-a342-490d-a22f-f90f67b47edd" containerName="revoke-keys"
Sep 09 12:45:00.170270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.170014    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba5a583a-3325-4577-8ebc-7a5ce0f261fb" containerName="main"
Sep 09 12:45:00.173222 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.173198    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx"]
Sep 09 12:45:00.173358 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.173339    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh"
Sep 09 12:45:00.176000 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.175972    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sk2jd\""
Sep 09 12:45:00.176555 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.176532    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx"
Sep 09 12:45:00.195946 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.195914    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh"]
Sep 09 12:45:00.196915 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.196771    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh"
Sep 09 12:45:00.201139 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.201111    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx"]
Sep 09 12:45:00.204425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.204398    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx"
Sep 09 12:45:00.475956 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.475915    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh"]
Sep 09 12:45:00.478776 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:45:00.478748    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cdbe28c_fff6_41d0_a84e_e678310a1ab8.slice/crio-5486334437a399a617c11dc6d1a5b4019326c7de1485a7246449d98866a0c7b4 WatchSource:0}: Error finding container 5486334437a399a617c11dc6d1a5b4019326c7de1485a7246449d98866a0c7b4: Status 404 returned error can't find the container with id 5486334437a399a617c11dc6d1a5b4019326c7de1485a7246449d98866a0c7b4
Sep 09 12:45:00.491479 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:00.491453    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx"]
Sep 09 12:45:00.494587 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:45:00.494561    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda203271b_1e83_432e_8e3f_f11675e414b8.slice/crio-bb2775729dc29ca126e797daebbe28cb0dc7bd1428f17da27e6edae18850c650 WatchSource:0}: Error finding container bb2775729dc29ca126e797daebbe28cb0dc7bd1428f17da27e6edae18850c650: Status 404 returned error can't find the container with id bb2775729dc29ca126e797daebbe28cb0dc7bd1428f17da27e6edae18850c650
Sep 09 12:45:01.216623 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:01.216535    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx" event={"ID":"a203271b-1e83-432e-8e3f-f11675e414b8","Type":"ContainerStarted","Data":"69fb9ef4062758754dfd26f7b21ef7ff003ff0add6bd6c023b436cfb68c2b11a"}
Sep 09 12:45:01.216623 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:01.216594    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx" event={"ID":"a203271b-1e83-432e-8e3f-f11675e414b8","Type":"ContainerStarted","Data":"bb2775729dc29ca126e797daebbe28cb0dc7bd1428f17da27e6edae18850c650"}
Sep 09 12:45:01.218269 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:01.218238    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh" event={"ID":"3cdbe28c-fff6-41d0-a84e-e678310a1ab8","Type":"ContainerStarted","Data":"f9c158027712380ec26f465f217f2f0f14c7f489475d776d1413997fc181a695"}
Sep 09 12:45:01.218435 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:01.218290    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh" event={"ID":"3cdbe28c-fff6-41d0-a84e-e678310a1ab8","Type":"ContainerStarted","Data":"5486334437a399a617c11dc6d1a5b4019326c7de1485a7246449d98866a0c7b4"}
Sep 09 12:45:01.233873 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:01.233823    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx" podStartSLOduration=1.233807625 podStartE2EDuration="1.233807625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:45:01.233572095 +0000 UTC m=+2045.875191300" watchObservedRunningTime="2026-09-09 12:45:01.233807625 +0000 UTC m=+2045.875426841"
Sep 09 12:45:01.251767 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:01.251672    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh" podStartSLOduration=1.251657598 podStartE2EDuration="1.251657598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:45:01.249754738 +0000 UTC m=+2045.891373943" watchObservedRunningTime="2026-09-09 12:45:01.251657598 +0000 UTC m=+2045.893276801"
Sep 09 12:45:02.225338 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:02.225306    2585 generic.go:356] "Generic (PLEG): container finished" podID="a203271b-1e83-432e-8e3f-f11675e414b8" containerID="69fb9ef4062758754dfd26f7b21ef7ff003ff0add6bd6c023b436cfb68c2b11a" exitCode=0
Sep 09 12:45:02.225830 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:02.225380    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx" event={"ID":"a203271b-1e83-432e-8e3f-f11675e414b8","Type":"ContainerDied","Data":"69fb9ef4062758754dfd26f7b21ef7ff003ff0add6bd6c023b436cfb68c2b11a"}
Sep 09 12:45:02.226821 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:02.226797    2585 generic.go:356] "Generic (PLEG): container finished" podID="3cdbe28c-fff6-41d0-a84e-e678310a1ab8" containerID="f9c158027712380ec26f465f217f2f0f14c7f489475d776d1413997fc181a695" exitCode=0
Sep 09 12:45:02.226929 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:02.226875    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh" event={"ID":"3cdbe28c-fff6-41d0-a84e-e678310a1ab8","Type":"ContainerDied","Data":"f9c158027712380ec26f465f217f2f0f14c7f489475d776d1413997fc181a695"}
Sep 09 12:45:03.375986 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:03.375961    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx"
Sep 09 12:45:03.412844 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:03.412815    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh"
Sep 09 12:45:04.239415 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:04.239326    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx"
Sep 09 12:45:04.239415 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:04.239333    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-dx5bx" event={"ID":"a203271b-1e83-432e-8e3f-f11675e414b8","Type":"ContainerDied","Data":"bb2775729dc29ca126e797daebbe28cb0dc7bd1428f17da27e6edae18850c650"}
Sep 09 12:45:04.239415 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:04.239387    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb2775729dc29ca126e797daebbe28cb0dc7bd1428f17da27e6edae18850c650"
Sep 09 12:45:04.240673 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:04.240653    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh"
Sep 09 12:45:04.240812 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:04.240671    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh" event={"ID":"3cdbe28c-fff6-41d0-a84e-e678310a1ab8","Type":"ContainerDied","Data":"5486334437a399a617c11dc6d1a5b4019326c7de1485a7246449d98866a0c7b4"}
Sep 09 12:45:04.240812 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:04.240734    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5486334437a399a617c11dc6d1a5b4019326c7de1485a7246449d98866a0c7b4"
Sep 09 12:45:25.036309 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:25.036213    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh"]
Sep 09 12:45:25.040622 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:25.040590    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-8ec58b-29815965-v9tdh"]
Sep 09 12:45:25.900450 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:25.900418    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cdbe28c-fff6-41d0-a84e-e678310a1ab8" path="/var/lib/kubelet/pods/3cdbe28c-fff6-41d0-a84e-e678310a1ab8/volumes"
Sep 09 12:45:32.360131 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:32.360092    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"]
Sep 09 12:45:32.360775 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:32.360357    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy" containerID="cri-o://abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a" gracePeriod=30
Sep 09 12:45:37.617405 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.617376    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:45:37.704571 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704487    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-data\") pod \"92f716ca-a856-48d6-b97d-e99807e70a82\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") "
Sep 09 12:45:37.704571 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704533    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-certs\") pod \"92f716ca-a856-48d6-b97d-e99807e70a82\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") "
Sep 09 12:45:37.704571 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704552    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92f716ca-a856-48d6-b97d-e99807e70a82-istiod-ca-cert\") pod \"92f716ca-a856-48d6-b97d-e99807e70a82\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") "
Sep 09 12:45:37.704868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704593    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-socket\") pod \"92f716ca-a856-48d6-b97d-e99807e70a82\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") "
Sep 09 12:45:37.704868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704621    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-envoy\") pod \"92f716ca-a856-48d6-b97d-e99807e70a82\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") "
Sep 09 12:45:37.704868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704667    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-istio-token\") pod \"92f716ca-a856-48d6-b97d-e99807e70a82\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") "
Sep 09 12:45:37.704868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704761    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-credential-socket\") pod \"92f716ca-a856-48d6-b97d-e99807e70a82\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") "
Sep 09 12:45:37.704868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704805    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92f716ca-a856-48d6-b97d-e99807e70a82-istio-podinfo\") pod \"92f716ca-a856-48d6-b97d-e99807e70a82\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") "
Sep 09 12:45:37.704868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704843    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqcp7\" (UniqueName: \"kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-kube-api-access-xqcp7\") pod \"92f716ca-a856-48d6-b97d-e99807e70a82\" (UID: \"92f716ca-a856-48d6-b97d-e99807e70a82\") "
Sep 09 12:45:37.704868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704847    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-data" (OuterVolumeSpecName: "istio-data") pod "92f716ca-a856-48d6-b97d-e99807e70a82" (UID: "92f716ca-a856-48d6-b97d-e99807e70a82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:37.704868 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704866    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92f716ca-a856-48d6-b97d-e99807e70a82" (UID: "92f716ca-a856-48d6-b97d-e99807e70a82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:37.705289 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.704973    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92f716ca-a856-48d6-b97d-e99807e70a82" (UID: "92f716ca-a856-48d6-b97d-e99807e70a82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:37.705289 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.705143    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92f716ca-a856-48d6-b97d-e99807e70a82" (UID: "92f716ca-a856-48d6-b97d-e99807e70a82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:37.705289 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.705232    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92f716ca-a856-48d6-b97d-e99807e70a82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92f716ca-a856-48d6-b97d-e99807e70a82" (UID: "92f716ca-a856-48d6-b97d-e99807e70a82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:37.705406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.705304    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:37.705406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.705327    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:37.705406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.705343    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:37.705406 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.705356    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:37.707114 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.707085    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92f716ca-a856-48d6-b97d-e99807e70a82" (UID: "92f716ca-a856-48d6-b97d-e99807e70a82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:37.707212 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.707192    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-istio-token" (OuterVolumeSpecName: "istio-token") pod "92f716ca-a856-48d6-b97d-e99807e70a82" (UID: "92f716ca-a856-48d6-b97d-e99807e70a82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:37.707464 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.707437    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92f716ca-a856-48d6-b97d-e99807e70a82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92f716ca-a856-48d6-b97d-e99807e70a82" (UID: "92f716ca-a856-48d6-b97d-e99807e70a82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:45:37.707464 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.707439    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-kube-api-access-xqcp7" (OuterVolumeSpecName: "kube-api-access-xqcp7") pod "92f716ca-a856-48d6-b97d-e99807e70a82" (UID: "92f716ca-a856-48d6-b97d-e99807e70a82"). InnerVolumeSpecName "kube-api-access-xqcp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:37.806214 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.806183    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92f716ca-a856-48d6-b97d-e99807e70a82-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:37.806214 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.806208    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:37.806214 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.806216    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92f716ca-a856-48d6-b97d-e99807e70a82-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:37.806214 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.806224    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqcp7\" (UniqueName: \"kubernetes.io/projected/92f716ca-a856-48d6-b97d-e99807e70a82-kube-api-access-xqcp7\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:37.806466 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:37.806233    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92f716ca-a856-48d6-b97d-e99807e70a82-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:38.438210 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.438166    2585 generic.go:356] "Generic (PLEG): container finished" podID="92f716ca-a856-48d6-b97d-e99807e70a82" containerID="abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a" exitCode=0
Sep 09 12:45:38.438356 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.438223    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"
Sep 09 12:45:38.438356 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.438249    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" event={"ID":"92f716ca-a856-48d6-b97d-e99807e70a82","Type":"ContainerDied","Data":"abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a"}
Sep 09 12:45:38.438356 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.438313    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq" event={"ID":"92f716ca-a856-48d6-b97d-e99807e70a82","Type":"ContainerDied","Data":"25644945ee02cae8348b5e2c2b4089d5bd8b732e1777cfacb25e026075e82ab4"}
Sep 09 12:45:38.438356 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.438335    2585 scope.go:117] "RemoveContainer" containerID="abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a"
Sep 09 12:45:38.447472 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.447454    2585 scope.go:117] "RemoveContainer" containerID="8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0"
Sep 09 12:45:38.455861 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.455751    2585 scope.go:117] "RemoveContainer" containerID="abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a"
Sep 09 12:45:38.456405 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:45:38.456365    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a\": container with ID starting with abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a not found: ID does not exist" containerID="abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a"
Sep 09 12:45:38.456520 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.456417    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a"} err="failed to get container status \"abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a\": rpc error: code = NotFound desc = could not find container \"abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a\": container with ID starting with abce21c2c3df3fd1176c329d5cd22cac23d6f9bee258f788ed2674d048543a9a not found: ID does not exist"
Sep 09 12:45:38.456520 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.456443    2585 scope.go:117] "RemoveContainer" containerID="8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0"
Sep 09 12:45:38.456800 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:45:38.456765    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0\": container with ID starting with 8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0 not found: ID does not exist" containerID="8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0"
Sep 09 12:45:38.456859 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.456804    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0"} err="failed to get container status \"8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0\": rpc error: code = NotFound desc = could not find container \"8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0\": container with ID starting with 8661f7b3b198af052eb1f2b5a79b315ad594997f0dad9a0158aafcd64992a7b0 not found: ID does not exist"
Sep 09 12:45:38.458551 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.458532    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"]
Sep 09 12:45:38.462659 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:38.462640    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-8ec58b-openshift-default-ddb498d5d-nhcrq"]
Sep 09 12:45:39.900108 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:39.900074    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" path="/var/lib/kubelet/pods/92f716ca-a856-48d6-b97d-e99807e70a82/volumes"
Sep 09 12:45:43.260272 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.260240    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"]
Sep 09 12:45:43.260693 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.260457    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm" podUID="a3ad359b-cd7a-46db-aca2-3de2aa4e189e" containerName="payload-processing" containerID="cri-o://be92b27d393e6f06f01c305fd4d2b23c39df02f21c7c7870424a0f9ce757d075" gracePeriod=30
Sep 09 12:45:43.468969 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.468935    2585 generic.go:356] "Generic (PLEG): container finished" podID="a3ad359b-cd7a-46db-aca2-3de2aa4e189e" containerID="be92b27d393e6f06f01c305fd4d2b23c39df02f21c7c7870424a0f9ce757d075" exitCode=0
Sep 09 12:45:43.469139 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.469005    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm" event={"ID":"a3ad359b-cd7a-46db-aca2-3de2aa4e189e","Type":"ContainerDied","Data":"be92b27d393e6f06f01c305fd4d2b23c39df02f21c7c7870424a0f9ce757d075"}
Sep 09 12:45:43.511414 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.511353    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:45:43.559302 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.559262    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsrsw\" (UniqueName: \"kubernetes.io/projected/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-kube-api-access-xsrsw\") pod \"a3ad359b-cd7a-46db-aca2-3de2aa4e189e\" (UID: \"a3ad359b-cd7a-46db-aca2-3de2aa4e189e\") "
Sep 09 12:45:43.559491 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.559316    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-plugins-config-volume\") pod \"a3ad359b-cd7a-46db-aca2-3de2aa4e189e\" (UID: \"a3ad359b-cd7a-46db-aca2-3de2aa4e189e\") "
Sep 09 12:45:43.559759 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.559737    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3ad359b-cd7a-46db-aca2-3de2aa4e189e" (UID: "a3ad359b-cd7a-46db-aca2-3de2aa4e189e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:43.561482 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.561454    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-kube-api-access-xsrsw" (OuterVolumeSpecName: "kube-api-access-xsrsw") pod "a3ad359b-cd7a-46db-aca2-3de2aa4e189e" (UID: "a3ad359b-cd7a-46db-aca2-3de2aa4e189e"). InnerVolumeSpecName "kube-api-access-xsrsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:43.660160 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.660131    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsrsw\" (UniqueName: \"kubernetes.io/projected/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-kube-api-access-xsrsw\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:43.660160 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:43.660157    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3ad359b-cd7a-46db-aca2-3de2aa4e189e-plugins-config-volume\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:44.476487 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:44.476455    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"
Sep 09 12:45:44.476910 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:44.476455    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm" event={"ID":"a3ad359b-cd7a-46db-aca2-3de2aa4e189e","Type":"ContainerDied","Data":"a876b38de14e36c951765f89572a2436957dd5616e5264f3f6a75f1592e95c57"}
Sep 09 12:45:44.476910 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:44.476592    2585 scope.go:117] "RemoveContainer" containerID="be92b27d393e6f06f01c305fd4d2b23c39df02f21c7c7870424a0f9ce757d075"
Sep 09 12:45:44.493540 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:44.493512    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"]
Sep 09 12:45:44.498309 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:44.498283    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-679f45-5bf7fc749c-69rpm"]
Sep 09 12:45:45.907433 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:45.907400    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3ad359b-cd7a-46db-aca2-3de2aa4e189e" path="/var/lib/kubelet/pods/a3ad359b-cd7a-46db-aca2-3de2aa4e189e/volumes"
Sep 09 12:45:50.793111 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:50.793070    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"]
Sep 09 12:45:50.793584 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:50.793319    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy" containerID="cri-o://c2f32df505e0375c97e118f0d9374fcf26438fcb3ee9a8853639abbd0b42afbf" gracePeriod=30
Sep 09 12:45:56.059581 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.059521    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:45:56.188080 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188044    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-data\") pod \"b517406b-c239-4ff5-837f-354d938c575b\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") "
Sep 09 12:45:56.188270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188101    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b517406b-c239-4ff5-837f-354d938c575b-istio-podinfo\") pod \"b517406b-c239-4ff5-837f-354d938c575b\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") "
Sep 09 12:45:56.188270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188129    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-istio-token\") pod \"b517406b-c239-4ff5-837f-354d938c575b\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") "
Sep 09 12:45:56.188270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188151    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b517406b-c239-4ff5-837f-354d938c575b-istiod-ca-cert\") pod \"b517406b-c239-4ff5-837f-354d938c575b\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") "
Sep 09 12:45:56.188270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188195    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7k5n\" (UniqueName: \"kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-kube-api-access-p7k5n\") pod \"b517406b-c239-4ff5-837f-354d938c575b\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") "
Sep 09 12:45:56.188270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188222    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-envoy\") pod \"b517406b-c239-4ff5-837f-354d938c575b\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") "
Sep 09 12:45:56.188270 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188244    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-data" (OuterVolumeSpecName: "istio-data") pod "b517406b-c239-4ff5-837f-354d938c575b" (UID: "b517406b-c239-4ff5-837f-354d938c575b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:56.188570 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188263    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-certs\") pod \"b517406b-c239-4ff5-837f-354d938c575b\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") "
Sep 09 12:45:56.188570 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188341    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-credential-socket\") pod \"b517406b-c239-4ff5-837f-354d938c575b\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") "
Sep 09 12:45:56.188570 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188437    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-socket\") pod \"b517406b-c239-4ff5-837f-354d938c575b\" (UID: \"b517406b-c239-4ff5-837f-354d938c575b\") "
Sep 09 12:45:56.188570 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188545    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b517406b-c239-4ff5-837f-354d938c575b" (UID: "b517406b-c239-4ff5-837f-354d938c575b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:56.188828 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188653    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b517406b-c239-4ff5-837f-354d938c575b" (UID: "b517406b-c239-4ff5-837f-354d938c575b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:56.188889 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188840    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b517406b-c239-4ff5-837f-354d938c575b" (UID: "b517406b-c239-4ff5-837f-354d938c575b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:56.188889 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188855    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:56.188889 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188877    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:56.189218 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188891    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:56.189218 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.188917    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b517406b-c239-4ff5-837f-354d938c575b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b517406b-c239-4ff5-837f-354d938c575b" (UID: "b517406b-c239-4ff5-837f-354d938c575b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:56.190689 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.190663    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b517406b-c239-4ff5-837f-354d938c575b" (UID: "b517406b-c239-4ff5-837f-354d938c575b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:56.190689 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.190667    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-kube-api-access-p7k5n" (OuterVolumeSpecName: "kube-api-access-p7k5n") pod "b517406b-c239-4ff5-837f-354d938c575b" (UID: "b517406b-c239-4ff5-837f-354d938c575b"). InnerVolumeSpecName "kube-api-access-p7k5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:56.190689 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.190669    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b517406b-c239-4ff5-837f-354d938c575b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b517406b-c239-4ff5-837f-354d938c575b" (UID: "b517406b-c239-4ff5-837f-354d938c575b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:45:56.190689 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.190689    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-istio-token" (OuterVolumeSpecName: "istio-token") pod "b517406b-c239-4ff5-837f-354d938c575b" (UID: "b517406b-c239-4ff5-837f-354d938c575b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:56.290064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.290024    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7k5n\" (UniqueName: \"kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-kube-api-access-p7k5n\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:56.290064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.290054    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:56.290064 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.290067    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b517406b-c239-4ff5-837f-354d938c575b-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:56.290346 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.290077    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b517406b-c239-4ff5-837f-354d938c575b-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:56.290346 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.290088    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b517406b-c239-4ff5-837f-354d938c575b-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:56.290346 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.290100    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b517406b-c239-4ff5-837f-354d938c575b-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:45:56.371247 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.371162    2585 scope.go:117] "RemoveContainer" containerID="a8504da3b674fa095952424d626c20a155c1d68616ded92580ee556b96d04408"
Sep 09 12:45:56.379975 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.379951    2585 scope.go:117] "RemoveContainer" containerID="cfc50fc3f5c118da1e9588ca60b59257625af2999381b423a1eb556964f3c3ca"
Sep 09 12:45:56.388158 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.388142    2585 scope.go:117] "RemoveContainer" containerID="c2f32df505e0375c97e118f0d9374fcf26438fcb3ee9a8853639abbd0b42afbf"
Sep 09 12:45:56.542194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.542157    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" event={"ID":"b517406b-c239-4ff5-837f-354d938c575b","Type":"ContainerDied","Data":"c2f32df505e0375c97e118f0d9374fcf26438fcb3ee9a8853639abbd0b42afbf"}
Sep 09 12:45:56.542194 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.542184    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"
Sep 09 12:45:56.542442 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.542212    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4" event={"ID":"b517406b-c239-4ff5-837f-354d938c575b","Type":"ContainerDied","Data":"68085e13f2d12732825e0195a5034cee213ce0d9ea68525b29b7a3004516f7bb"}
Sep 09 12:45:56.581431 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.581396    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"]
Sep 09 12:45:56.585884 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:56.585856    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-679f45-openshift-default-84696cbc98-gklj4"]
Sep 09 12:45:57.900800 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:45:57.900769    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b517406b-c239-4ff5-837f-354d938c575b" path="/var/lib/kubelet/pods/b517406b-c239-4ff5-837f-354d938c575b/volumes"
Sep 09 12:46:09.257368 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:09.257334    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"]
Sep 09 12:46:09.258338 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:09.258302    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy" containerID="cri-o://629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638" gracePeriod=30
Sep 09 12:46:14.511955 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.511930    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:46:14.563054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.562953    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-envoy\") pod \"a1d50c73-3bdd-4c5e-8948-81daf607b301\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") "
Sep 09 12:46:14.563054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563014    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-certs\") pod \"a1d50c73-3bdd-4c5e-8948-81daf607b301\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") "
Sep 09 12:46:14.563054 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563039    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-data\") pod \"a1d50c73-3bdd-4c5e-8948-81daf607b301\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") "
Sep 09 12:46:14.563347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563080    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-credential-socket\") pod \"a1d50c73-3bdd-4c5e-8948-81daf607b301\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") "
Sep 09 12:46:14.563347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563110    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-token\") pod \"a1d50c73-3bdd-4c5e-8948-81daf607b301\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") "
Sep 09 12:46:14.563347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563159    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxjmb\" (UniqueName: \"kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-kube-api-access-rxjmb\") pod \"a1d50c73-3bdd-4c5e-8948-81daf607b301\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") "
Sep 09 12:46:14.563347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563206    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-podinfo\") pod \"a1d50c73-3bdd-4c5e-8948-81daf607b301\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") "
Sep 09 12:46:14.563347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563237    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d50c73-3bdd-4c5e-8948-81daf607b301-istiod-ca-cert\") pod \"a1d50c73-3bdd-4c5e-8948-81daf607b301\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") "
Sep 09 12:46:14.563347 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563276    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-socket\") pod \"a1d50c73-3bdd-4c5e-8948-81daf607b301\" (UID: \"a1d50c73-3bdd-4c5e-8948-81daf607b301\") "
Sep 09 12:46:14.563651 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563409    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-data" (OuterVolumeSpecName: "istio-data") pod "a1d50c73-3bdd-4c5e-8948-81daf607b301" (UID: "a1d50c73-3bdd-4c5e-8948-81daf607b301"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:46:14.563651 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563426    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a1d50c73-3bdd-4c5e-8948-81daf607b301" (UID: "a1d50c73-3bdd-4c5e-8948-81daf607b301"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:46:14.563651 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563642    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-data\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:46:14.563828 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563660    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-credential-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:46:14.563876 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.563827    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1d50c73-3bdd-4c5e-8948-81daf607b301-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a1d50c73-3bdd-4c5e-8948-81daf607b301" (UID: "a1d50c73-3bdd-4c5e-8948-81daf607b301"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:46:14.564250 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.564215    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a1d50c73-3bdd-4c5e-8948-81daf607b301" (UID: "a1d50c73-3bdd-4c5e-8948-81daf607b301"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:46:14.564522 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.564300    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a1d50c73-3bdd-4c5e-8948-81daf607b301" (UID: "a1d50c73-3bdd-4c5e-8948-81daf607b301"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:46:14.565508 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.565480    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a1d50c73-3bdd-4c5e-8948-81daf607b301" (UID: "a1d50c73-3bdd-4c5e-8948-81daf607b301"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:46:14.566111 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.566091    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-kube-api-access-rxjmb" (OuterVolumeSpecName: "kube-api-access-rxjmb") pod "a1d50c73-3bdd-4c5e-8948-81daf607b301" (UID: "a1d50c73-3bdd-4c5e-8948-81daf607b301"). InnerVolumeSpecName "kube-api-access-rxjmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:46:14.566235 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.566210    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-token" (OuterVolumeSpecName: "istio-token") pod "a1d50c73-3bdd-4c5e-8948-81daf607b301" (UID: "a1d50c73-3bdd-4c5e-8948-81daf607b301"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:46:14.566308 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.566274    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a1d50c73-3bdd-4c5e-8948-81daf607b301" (UID: "a1d50c73-3bdd-4c5e-8948-81daf607b301"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:46:14.643518 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.643484    2585 generic.go:356] "Generic (PLEG): container finished" podID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerID="629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638" exitCode=0
Sep 09 12:46:14.643694 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.643543    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"
Sep 09 12:46:14.643694 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.643576    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" event={"ID":"a1d50c73-3bdd-4c5e-8948-81daf607b301","Type":"ContainerDied","Data":"629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638"}
Sep 09 12:46:14.643694 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.643638    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r" event={"ID":"a1d50c73-3bdd-4c5e-8948-81daf607b301","Type":"ContainerDied","Data":"575c48354caf0c358589022f7cf241d7040da2fda5b2ed7357a80d45a8b84bdb"}
Sep 09 12:46:14.643694 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.643662    2585 scope.go:117] "RemoveContainer" containerID="629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638"
Sep 09 12:46:14.660055 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.660033    2585 scope.go:117] "RemoveContainer" containerID="7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5"
Sep 09 12:46:14.664634 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.664610    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:46:14.664634 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.664633    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-token\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:46:14.664831 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.664642    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxjmb\" (UniqueName: \"kubernetes.io/projected/a1d50c73-3bdd-4c5e-8948-81daf607b301-kube-api-access-rxjmb\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:46:14.664831 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.664651    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-podinfo\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:46:14.664831 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.664661    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d50c73-3bdd-4c5e-8948-81daf607b301-istiod-ca-cert\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:46:14.664831 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.664668    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-workload-socket\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:46:14.664831 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.664676    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d50c73-3bdd-4c5e-8948-81daf607b301-istio-envoy\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:46:14.668719 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.668681    2585 scope.go:117] "RemoveContainer" containerID="629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638"
Sep 09 12:46:14.669156 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:46:14.669131    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638\": container with ID starting with 629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638 not found: ID does not exist" containerID="629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638"
Sep 09 12:46:14.669225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.669164    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638"} err="failed to get container status \"629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638\": rpc error: code = NotFound desc = could not find container \"629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638\": container with ID starting with 629553b27a63a7424fef34079d8bf1cd2a0a5808b02bef9a54b5fbb2c5342638 not found: ID does not exist"
Sep 09 12:46:14.669225 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.669180    2585 scope.go:117] "RemoveContainer" containerID="7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5"
Sep 09 12:46:14.669491 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:46:14.669463    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5\": container with ID starting with 7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5 not found: ID does not exist" containerID="7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5"
Sep 09 12:46:14.669553 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.669502    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5"} err="failed to get container status \"7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5\": rpc error: code = NotFound desc = could not find container \"7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5\": container with ID starting with 7f9d7e983354f45c400081b2b0985fdc3faae205ffd90e537aa542f96bc15dc5 not found: ID does not exist"
Sep 09 12:46:14.677900 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.677871    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"]
Sep 09 12:46:14.684070 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:14.684049    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cfcdfd-openshift-default-85f76bb874-9cb7r"]
Sep 09 12:46:15.899845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:46:15.899813    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" path="/var/lib/kubelet/pods/a1d50c73-3bdd-4c5e-8948-81daf607b301/volumes"
Sep 09 12:48:52.586682 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:52.586644    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"]
Sep 09 12:48:52.587292 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:52.586848    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn" podUID="dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc" containerName="manager" containerID="cri-o://f1a1c09ab5f3b63b295a17cb1b855235abd6770aaf3c412783dd43c6eb4d09d9" gracePeriod=10
Sep 09 12:48:54.470500 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:54.470419    2585 generic.go:356] "Generic (PLEG): container finished" podID="dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc" containerID="f1a1c09ab5f3b63b295a17cb1b855235abd6770aaf3c412783dd43c6eb4d09d9" exitCode=0
Sep 09 12:48:54.470860 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:54.470494    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn" event={"ID":"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc","Type":"ContainerDied","Data":"f1a1c09ab5f3b63b295a17cb1b855235abd6770aaf3c412783dd43c6eb4d09d9"}
Sep 09 12:48:54.831973 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:54.831948    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:48:54.949473 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:54.949427    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-extensions-socket-volume\") pod \"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc\" (UID: \"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc\") "
Sep 09 12:48:54.949647 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:54.949519    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjfzt\" (UniqueName: \"kubernetes.io/projected/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-kube-api-access-rjfzt\") pod \"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc\" (UID: \"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc\") "
Sep 09 12:48:54.949805 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:54.949782    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc" (UID: "dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:48:54.957598 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:54.957558    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-kube-api-access-rjfzt" (OuterVolumeSpecName: "kube-api-access-rjfzt") pod "dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc" (UID: "dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc"). InnerVolumeSpecName "kube-api-access-rjfzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:48:55.050864 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:55.050834    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-extensions-socket-volume\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:48:55.050864 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:55.050859    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjfzt\" (UniqueName: \"kubernetes.io/projected/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc-kube-api-access-rjfzt\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:48:55.477013 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:55.476927    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn" event={"ID":"dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc","Type":"ContainerDied","Data":"7a0ebc01abfb4517871e6b82040f6b567ecec31016ecae2ad4b4131c2f79cbbf"}
Sep 09 12:48:55.477418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:55.477018    2585 scope.go:117] "RemoveContainer" containerID="f1a1c09ab5f3b63b295a17cb1b855235abd6770aaf3c412783dd43c6eb4d09d9"
Sep 09 12:48:55.477418 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:55.476936    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"
Sep 09 12:48:55.499948 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:55.499920    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"]
Sep 09 12:48:55.503877 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:55.503855    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sphfn"]
Sep 09 12:48:55.899769 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:48:55.899730    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc" path="/var/lib/kubelet/pods/dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc/volumes"
Sep 09 12:49:53.292385 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.292338    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"]
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.292911    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc" containerName="manager"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.292931    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc" containerName="manager"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.292947    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.292955    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.292976    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.292986    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.292995    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3ad359b-cd7a-46db-aca2-3de2aa4e189e" containerName="payload-processing"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293002    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3ad359b-cd7a-46db-aca2-3de2aa4e189e" containerName="payload-processing"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293011    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293016    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293025    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cdbe28c-fff6-41d0-a84e-e678310a1ab8" containerName="cleanup"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293030    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cdbe28c-fff6-41d0-a84e-e678310a1ab8" containerName="cleanup"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293042    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293048    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293053    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293058    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy"
Sep 09 12:49:53.293058 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293066    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a203271b-1e83-432e-8e3f-f11675e414b8" containerName="cleanup"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293071    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a203271b-1e83-432e-8e3f-f11675e414b8" containerName="cleanup"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293085    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293091    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293157    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3ad359b-cd7a-46db-aca2-3de2aa4e189e" containerName="payload-processing"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293165    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cdbe28c-fff6-41d0-a84e-e678310a1ab8" containerName="cleanup"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293172    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a203271b-1e83-432e-8e3f-f11675e414b8" containerName="cleanup"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293177    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293182    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293186    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293193    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="92f716ca-a856-48d6-b97d-e99807e70a82" containerName="istio-proxy"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293198    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda363b8-fb6b-4e7b-b7ef-91b4c2315bbc" containerName="manager"
Sep 09 12:49:53.293587 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.293202    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b517406b-c239-4ff5-837f-354d938c575b" containerName="istio-proxy"
Sep 09 12:49:53.296224 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.296202    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"
Sep 09 12:49:53.298856 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.298828    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mc7gz\""
Sep 09 12:49:53.308081 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.308058    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"]
Sep 09 12:49:53.382565 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.382527    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d654f50b-101e-4eef-ad46-48a8db83a8e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v982w\" (UID: \"d654f50b-101e-4eef-ad46-48a8db83a8e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"
Sep 09 12:49:53.382763 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.382613    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdxdz\" (UniqueName: \"kubernetes.io/projected/d654f50b-101e-4eef-ad46-48a8db83a8e3-kube-api-access-zdxdz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v982w\" (UID: \"d654f50b-101e-4eef-ad46-48a8db83a8e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"
Sep 09 12:49:53.483609 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.483566    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d654f50b-101e-4eef-ad46-48a8db83a8e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v982w\" (UID: \"d654f50b-101e-4eef-ad46-48a8db83a8e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"
Sep 09 12:49:53.483834 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.483641    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdxdz\" (UniqueName: \"kubernetes.io/projected/d654f50b-101e-4eef-ad46-48a8db83a8e3-kube-api-access-zdxdz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v982w\" (UID: \"d654f50b-101e-4eef-ad46-48a8db83a8e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"
Sep 09 12:49:53.484007 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.483986    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d654f50b-101e-4eef-ad46-48a8db83a8e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v982w\" (UID: \"d654f50b-101e-4eef-ad46-48a8db83a8e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"
Sep 09 12:49:53.495224 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.495197    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdxdz\" (UniqueName: \"kubernetes.io/projected/d654f50b-101e-4eef-ad46-48a8db83a8e3-kube-api-access-zdxdz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v982w\" (UID: \"d654f50b-101e-4eef-ad46-48a8db83a8e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"
Sep 09 12:49:53.606385 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.606302    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"
Sep 09 12:49:53.893294 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:53.893269    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"]
Sep 09 12:49:53.896012 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:49:53.895988    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd654f50b_101e_4eef_ad46_48a8db83a8e3.slice/crio-e8bc6a8793394453479a593e5b8c8e41531ca2ce424e0d9c77cbfd1adbc42d34 WatchSource:0}: Error finding container e8bc6a8793394453479a593e5b8c8e41531ca2ce424e0d9c77cbfd1adbc42d34: Status 404 returned error can't find the container with id e8bc6a8793394453479a593e5b8c8e41531ca2ce424e0d9c77cbfd1adbc42d34
Sep 09 12:49:54.785066 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:54.785029    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w" event={"ID":"d654f50b-101e-4eef-ad46-48a8db83a8e3","Type":"ContainerStarted","Data":"b4ae9a1333a55f46ab997be365053f828f188efb48b74c0a1d704d52fbd93eb8"}
Sep 09 12:49:54.785445 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:54.785087    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"
Sep 09 12:49:54.785445 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:54.785104    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w" event={"ID":"d654f50b-101e-4eef-ad46-48a8db83a8e3","Type":"ContainerStarted","Data":"e8bc6a8793394453479a593e5b8c8e41531ca2ce424e0d9c77cbfd1adbc42d34"}
Sep 09 12:49:54.812593 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:54.812534    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w" podStartSLOduration=1.812518768 podStartE2EDuration="1.812518768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:54.811008422 +0000 UTC m=+2339.452627623" watchObservedRunningTime="2026-09-09 12:49:54.812518768 +0000 UTC m=+2339.454137972"
Sep 09 12:49:56.451123 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:49:56.451091    2585 scope.go:117] "RemoveContainer" containerID="703036c81fb83a7b27a6747e3475ebf694abf129aa066fb4055a8684b36bf1b7"
Sep 09 12:50:05.792246 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:50:05.792204    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v982w"
Sep 09 12:51:00.489630 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.489593    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"]
Sep 09 12:51:00.490123 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.490064    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1d50c73-3bdd-4c5e-8948-81daf607b301" containerName="istio-proxy"
Sep 09 12:51:00.493353 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.493332    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.495651 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.495625    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-a470f520-kserve-self-signed-certs\""
Sep 09 12:51:00.502934 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.502909    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"]
Sep 09 12:51:00.624504 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.624464    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-dshm\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.624504 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.624506    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee8192b-6f66-43da-995e-5658a87f2558-tls-certs\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.624747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.624537    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-model-cache\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.624747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.624597    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-tmp-dir\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.624747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.624672    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-home\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.624747 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.624742    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvpc8\" (UniqueName: \"kubernetes.io/projected/9ee8192b-6f66-43da-995e-5658a87f2558-kube-api-access-mvpc8\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.725860 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.725824    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvpc8\" (UniqueName: \"kubernetes.io/projected/9ee8192b-6f66-43da-995e-5658a87f2558-kube-api-access-mvpc8\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.726060 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.725882    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-dshm\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.726060 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.725900    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee8192b-6f66-43da-995e-5658a87f2558-tls-certs\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.726060 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.725927    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-model-cache\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.726060 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.725963    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-tmp-dir\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.726060 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.726013    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-home\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.726359 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.726319    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-model-cache\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.726397 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.726379    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-tmp-dir\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.726434 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.726406    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-home\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.728363 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.728342    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-dshm\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.728543 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.728522    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee8192b-6f66-43da-995e-5658a87f2558-tls-certs\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.737579 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.737555    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvpc8\" (UniqueName: \"kubernetes.io/projected/9ee8192b-6f66-43da-995e-5658a87f2558-kube-api-access-mvpc8\") pod \"e2e-same-id-b-a470f520-kserve-975f66d79-694h7\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") " pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:00.804079 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:00.804048    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:01.075896 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:01.075871    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"]
Sep 09 12:51:01.078303 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:51:01.078275    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ee8192b_6f66_43da_995e_5658a87f2558.slice/crio-4284de44789a4cb7448cabcf14215f0f05cefe76e47775e00a5212c9c2425439 WatchSource:0}: Error finding container 4284de44789a4cb7448cabcf14215f0f05cefe76e47775e00a5212c9c2425439: Status 404 returned error can't find the container with id 4284de44789a4cb7448cabcf14215f0f05cefe76e47775e00a5212c9c2425439
Sep 09 12:51:01.123603 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:01.123580    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7" event={"ID":"9ee8192b-6f66-43da-995e-5658a87f2558","Type":"ContainerStarted","Data":"4284de44789a4cb7448cabcf14215f0f05cefe76e47775e00a5212c9c2425439"}
Sep 09 12:51:02.130159 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:02.130119    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7" event={"ID":"9ee8192b-6f66-43da-995e-5658a87f2558","Type":"ContainerStarted","Data":"6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0"}
Sep 09 12:51:02.150925 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:02.150873    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7" podStartSLOduration=2.150856758 podStartE2EDuration="2.150856758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:51:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:51:02.149664973 +0000 UTC m=+2406.791284177" watchObservedRunningTime="2026-09-09 12:51:02.150856758 +0000 UTC m=+2406.792476022"
Sep 09 12:51:10.805180 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:10.805136    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:10.805670 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:10.805201    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:10.818209 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:10.818184    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:11.187560 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:11.187467    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:29.692599 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:29.692508    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"]
Sep 09 12:51:44.713213 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:44.713126    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7" podUID="9ee8192b-6f66-43da-995e-5658a87f2558" containerName="main" containerID="cri-o://6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0" gracePeriod=45
Sep 09 12:51:44.953307 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:44.953277    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:45.064635 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.064600    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee8192b-6f66-43da-995e-5658a87f2558-tls-certs\") pod \"9ee8192b-6f66-43da-995e-5658a87f2558\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") "
Sep 09 12:51:45.064860 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.064643    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvpc8\" (UniqueName: \"kubernetes.io/projected/9ee8192b-6f66-43da-995e-5658a87f2558-kube-api-access-mvpc8\") pod \"9ee8192b-6f66-43da-995e-5658a87f2558\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") "
Sep 09 12:51:45.064860 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.064663    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-dshm\") pod \"9ee8192b-6f66-43da-995e-5658a87f2558\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") "
Sep 09 12:51:45.064860 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.064838    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-tmp-dir\") pod \"9ee8192b-6f66-43da-995e-5658a87f2558\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") "
Sep 09 12:51:45.065035 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.064897    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-model-cache\") pod \"9ee8192b-6f66-43da-995e-5658a87f2558\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") "
Sep 09 12:51:45.065035 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.064953    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-home\") pod \"9ee8192b-6f66-43da-995e-5658a87f2558\" (UID: \"9ee8192b-6f66-43da-995e-5658a87f2558\") "
Sep 09 12:51:45.065149 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.065118    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9ee8192b-6f66-43da-995e-5658a87f2558" (UID: "9ee8192b-6f66-43da-995e-5658a87f2558"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:51:45.065201 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.065145    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-model-cache" (OuterVolumeSpecName: "model-cache") pod "9ee8192b-6f66-43da-995e-5658a87f2558" (UID: "9ee8192b-6f66-43da-995e-5658a87f2558"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:51:45.065344 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.065321    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-home" (OuterVolumeSpecName: "home") pod "9ee8192b-6f66-43da-995e-5658a87f2558" (UID: "9ee8192b-6f66-43da-995e-5658a87f2558"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:51:45.065489 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.065338    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-tmp-dir\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:51:45.065489 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.065358    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-model-cache\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:51:45.067197 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.067173    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee8192b-6f66-43da-995e-5658a87f2558-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ee8192b-6f66-43da-995e-5658a87f2558" (UID: "9ee8192b-6f66-43da-995e-5658a87f2558"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:51:45.067300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.067176    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-dshm" (OuterVolumeSpecName: "dshm") pod "9ee8192b-6f66-43da-995e-5658a87f2558" (UID: "9ee8192b-6f66-43da-995e-5658a87f2558"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:51:45.067300 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.067245    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee8192b-6f66-43da-995e-5658a87f2558-kube-api-access-mvpc8" (OuterVolumeSpecName: "kube-api-access-mvpc8") pod "9ee8192b-6f66-43da-995e-5658a87f2558" (UID: "9ee8192b-6f66-43da-995e-5658a87f2558"). InnerVolumeSpecName "kube-api-access-mvpc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:51:45.165952 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.165916    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-home\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:51:45.165952 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.165945    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee8192b-6f66-43da-995e-5658a87f2558-tls-certs\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:51:45.165952 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.165955    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvpc8\" (UniqueName: \"kubernetes.io/projected/9ee8192b-6f66-43da-995e-5658a87f2558-kube-api-access-mvpc8\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:51:45.166198 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.165963    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee8192b-6f66-43da-995e-5658a87f2558-dshm\") on node \"ip-10-0-131-207.ec2.internal\" DevicePath \"\""
Sep 09 12:51:45.351616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.351534    2585 generic.go:356] "Generic (PLEG): container finished" podID="9ee8192b-6f66-43da-995e-5658a87f2558" containerID="6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0" exitCode=0
Sep 09 12:51:45.351616 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.351581    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"
Sep 09 12:51:45.351891 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.351582    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7" event={"ID":"9ee8192b-6f66-43da-995e-5658a87f2558","Type":"ContainerDied","Data":"6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0"}
Sep 09 12:51:45.351891 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.351738    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7" event={"ID":"9ee8192b-6f66-43da-995e-5658a87f2558","Type":"ContainerDied","Data":"4284de44789a4cb7448cabcf14215f0f05cefe76e47775e00a5212c9c2425439"}
Sep 09 12:51:45.351891 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.351775    2585 scope.go:117] "RemoveContainer" containerID="6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0"
Sep 09 12:51:45.361668 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.361638    2585 scope.go:117] "RemoveContainer" containerID="6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0"
Sep 09 12:51:45.361929 ip-10-0-131-207 kubenswrapper[2585]: E0909 12:51:45.361910    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0\": container with ID starting with 6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0 not found: ID does not exist" containerID="6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0"
Sep 09 12:51:45.361981 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.361939    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0"} err="failed to get container status \"6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0\": rpc error: code = NotFound desc = could not find container \"6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0\": container with ID starting with 6d8b2de9a0136385a2dbdd6e50f903914f1d87f311cd8200a4db91b4ce5869a0 not found: ID does not exist"
Sep 09 12:51:45.374663 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.374637    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"]
Sep 09 12:51:45.379643 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.379623    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-a470f520-kserve-975f66d79-694h7"]
Sep 09 12:51:45.899671 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:45.899637    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ee8192b-6f66-43da-995e-5658a87f2558" path="/var/lib/kubelet/pods/9ee8192b-6f66-43da-995e-5658a87f2558/volumes"
Sep 09 12:51:56.484969 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:51:56.484933    2585 scope.go:117] "RemoveContainer" containerID="f9c158027712380ec26f465f217f2f0f14c7f489475d776d1413997fc181a695"
Sep 09 12:52:00.574552 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.574513    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"]
Sep 09 12:52:00.574960 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.574930    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee8192b-6f66-43da-995e-5658a87f2558" containerName="main"
Sep 09 12:52:00.574960 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.574943    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee8192b-6f66-43da-995e-5658a87f2558" containerName="main"
Sep 09 12:52:00.575032 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.575015    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ee8192b-6f66-43da-995e-5658a87f2558" containerName="main"
Sep 09 12:52:00.579425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.579406    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.582567 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.582534    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 12:52:00.582567 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.582542    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:52:00.582789 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.582572    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 12:52:00.582789 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.582595    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qz8xf\""
Sep 09 12:52:00.582789 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.582612    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 12:52:00.593138 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.593112    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"]
Sep 09 12:52:00.715277 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.715243    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba7b88e1-8173-4d39-ad42-51e7afd461de-metrics-certs\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.715451 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.715354    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba7b88e1-8173-4d39-ad42-51e7afd461de-openshift-service-ca\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.715451 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.715383    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpknt\" (UniqueName: \"kubernetes.io/projected/ba7b88e1-8173-4d39-ad42-51e7afd461de-kube-api-access-lpknt\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.715451 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.715439    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba7b88e1-8173-4d39-ad42-51e7afd461de-maas-api-tls\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.816171 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.816128    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba7b88e1-8173-4d39-ad42-51e7afd461de-maas-api-tls\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.816345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.816181    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba7b88e1-8173-4d39-ad42-51e7afd461de-metrics-certs\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.816345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.816249    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba7b88e1-8173-4d39-ad42-51e7afd461de-openshift-service-ca\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.816345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.816280    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpknt\" (UniqueName: \"kubernetes.io/projected/ba7b88e1-8173-4d39-ad42-51e7afd461de-kube-api-access-lpknt\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.817051 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.816966    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba7b88e1-8173-4d39-ad42-51e7afd461de-openshift-service-ca\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.818929 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.818906    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba7b88e1-8173-4d39-ad42-51e7afd461de-maas-api-tls\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.819039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.818906    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba7b88e1-8173-4d39-ad42-51e7afd461de-metrics-certs\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.824908 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.824864    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpknt\" (UniqueName: \"kubernetes.io/projected/ba7b88e1-8173-4d39-ad42-51e7afd461de-kube-api-access-lpknt\") pod \"maas-api-849567d9dc-zlqmc\" (UID: \"ba7b88e1-8173-4d39-ad42-51e7afd461de\") " pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:00.890961 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:00.890927    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:01.164995 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:01.164964    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"]
Sep 09 12:52:01.165966 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:52:01.165925    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba7b88e1_8173_4d39_ad42_51e7afd461de.slice/crio-ed879ff582cbb5ba06e0640f4b42948a15a060b61410b6908d3c37507d55e463 WatchSource:0}: Error finding container ed879ff582cbb5ba06e0640f4b42948a15a060b61410b6908d3c37507d55e463: Status 404 returned error can't find the container with id ed879ff582cbb5ba06e0640f4b42948a15a060b61410b6908d3c37507d55e463
Sep 09 12:52:01.167483 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:01.167465    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:52:01.436944 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:01.436854    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc" event={"ID":"ba7b88e1-8173-4d39-ad42-51e7afd461de","Type":"ContainerStarted","Data":"ed879ff582cbb5ba06e0640f4b42948a15a060b61410b6908d3c37507d55e463"}
Sep 09 12:52:04.457084 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:04.456983    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc" event={"ID":"ba7b88e1-8173-4d39-ad42-51e7afd461de","Type":"ContainerStarted","Data":"d63ec13a66796ed0931d024d0ffdf408daa24f1c4b51adb02d84fc80b02f4f10"}
Sep 09 12:52:04.457084 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:04.457064    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:04.475017 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:04.474966    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc" podStartSLOduration=1.560011917 podStartE2EDuration="4.47494743s" totalImagesPullingTime="2.914935513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:52:04.473269912 +0000 UTC m=+2469.114889118" watchObservedRunningTime="2026-09-09 12:52:04.47494743 +0000 UTC m=+2469.116566662"
Sep 09 12:52:10.468536 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:10.468506    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-849567d9dc-zlqmc"
Sep 09 12:52:29.988408 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:29.988379    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-92v57_325079e6-e247-4adc-b0e9-a37fae9d7b57/manager/0.log"
Sep 09 12:52:30.759551 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:30.759520    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk_1de04984-4073-43bc-bd71-33e8156d5527/copy-manifests/0.log"
Sep 09 12:52:30.767030 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:30.767008    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk_1de04984-4073-43bc-bd71-33e8156d5527/manager/0.log"
Sep 09 12:52:31.352302 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:31.352268    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-849567d9dc-zlqmc_ba7b88e1-8173-4d39-ad42-51e7afd461de/maas-api/0.log"
Sep 09 12:52:31.469236 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:31.469206    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29815950-gmqlp_647ff576-cf6d-4936-a267-4f6611c55ed8/cleanup/0.log"
Sep 09 12:52:31.579132 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:31.579103    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29815965-dx5bx_a203271b-1e83-432e-8e3f-f11675e414b8/cleanup/0.log"
Sep 09 12:52:32.622914 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:32.622881    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-x6mkb_c54e1303-c5a5-40cc-ad25-43165a3cb13e/manager/0.log"
Sep 09 12:52:32.740845 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:32.740812    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-97mrz_1cc91038-30ea-44f5-b002-0fb514b7eba6/manager/0.log"
Sep 09 12:52:32.983936 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:32.983904    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-v982w_d654f50b-101e-4eef-ad46-48a8db83a8e3/manager/0.log"
Sep 09 12:52:33.240691 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:33.240609    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-g5fwd_4aae3551-7e56-455c-aadc-d58e489cb63a/manager/0.log"
Sep 09 12:52:33.607754 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:33.607725    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp_d9279200-2465-4063-ae1f-59d2116f018a/istio-proxy/0.log"
Sep 09 12:52:34.327511 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:34.327474    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-chp7l_7b6e2ff7-9e23-49c1-aa14-b29047aeef39/payload-processing/0.log"
Sep 09 12:52:34.928008 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:34.927978    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-j6gzd_575ca116-ef49-422e-b08d-4538a4c11491/main/0.log"
Sep 09 12:52:35.417170 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:35.417146    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-x5q8w_3dc61d5e-2f57-4861-a1fa-be6fbf5c8d75/main/0.log"
Sep 09 12:52:44.034015 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:44.033984    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-m6ssq_a18b39f3-278e-4cc9-97b0-867a633220f3/global-pull-secret-syncer/0.log"
Sep 09 12:52:44.164356 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:44.164324    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vt2db_9ce58793-93d8-4b04-af82-66521edd67dc/konnectivity-agent/0.log"
Sep 09 12:52:44.214963 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:44.214927    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-207.ec2.internal_ba7b0232134df51badacb252b3d6af22/haproxy/0.log"
Sep 09 12:52:48.079582 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:48.079507    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-x6mkb_c54e1303-c5a5-40cc-ad25-43165a3cb13e/manager/0.log"
Sep 09 12:52:48.143252 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:48.143221    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-97mrz_1cc91038-30ea-44f5-b002-0fb514b7eba6/manager/0.log"
Sep 09 12:52:48.250242 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:48.250213    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-v982w_d654f50b-101e-4eef-ad46-48a8db83a8e3/manager/0.log"
Sep 09 12:52:48.345437 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:48.345341    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-g5fwd_4aae3551-7e56-455c-aadc-d58e489cb63a/manager/0.log"
Sep 09 12:52:50.073118 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.073092    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_870b927b-7eea-4d47-acc6-aef50a4d7d99/alertmanager/0.log"
Sep 09 12:52:50.095167 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.095139    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_870b927b-7eea-4d47-acc6-aef50a4d7d99/config-reloader/0.log"
Sep 09 12:52:50.118133 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.118106    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_870b927b-7eea-4d47-acc6-aef50a4d7d99/kube-rbac-proxy-web/0.log"
Sep 09 12:52:50.140129 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.140103    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_870b927b-7eea-4d47-acc6-aef50a4d7d99/kube-rbac-proxy/0.log"
Sep 09 12:52:50.164992 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.164969    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_870b927b-7eea-4d47-acc6-aef50a4d7d99/kube-rbac-proxy-metric/0.log"
Sep 09 12:52:50.187127 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.187067    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_870b927b-7eea-4d47-acc6-aef50a4d7d99/prom-label-proxy/0.log"
Sep 09 12:52:50.212290 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.212262    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_870b927b-7eea-4d47-acc6-aef50a4d7d99/init-config-reloader/0.log"
Sep 09 12:52:50.285089 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.285062    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-tlvc6_231e8ac2-4546-4653-923a-4da453187e31/kube-state-metrics/0.log"
Sep 09 12:52:50.306922 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.306888    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-tlvc6_231e8ac2-4546-4653-923a-4da453187e31/kube-rbac-proxy-main/0.log"
Sep 09 12:52:50.330636 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.330608    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-tlvc6_231e8ac2-4546-4653-923a-4da453187e31/kube-rbac-proxy-self/0.log"
Sep 09 12:52:50.363728 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.363684    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-b8fcf97b4-27xgf_f8d64d62-3e0b-4dc9-9244-685de948488a/metrics-server/0.log"
Sep 09 12:52:50.607399 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.607368    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wgc84_f85167fb-19a2-49b2-83fe-bdc865773943/node-exporter/0.log"
Sep 09 12:52:50.627990 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.627956    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wgc84_f85167fb-19a2-49b2-83fe-bdc865773943/kube-rbac-proxy/0.log"
Sep 09 12:52:50.649345 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.649319    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wgc84_f85167fb-19a2-49b2-83fe-bdc865773943/init-textfile/0.log"
Sep 09 12:52:50.676288 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.676262    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-br2tn_af02d5ca-a5b8-4588-9d21-e43935461318/kube-rbac-proxy-main/0.log"
Sep 09 12:52:50.701789 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.701761    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-br2tn_af02d5ca-a5b8-4588-9d21-e43935461318/kube-rbac-proxy-self/0.log"
Sep 09 12:52:50.721236 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.721210    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-br2tn_af02d5ca-a5b8-4588-9d21-e43935461318/openshift-state-metrics/0.log"
Sep 09 12:52:50.758987 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.758964    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62b29a26-2b25-4eaa-965e-5f09ed467f11/prometheus/0.log"
Sep 09 12:52:50.781103 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.781080    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62b29a26-2b25-4eaa-965e-5f09ed467f11/config-reloader/0.log"
Sep 09 12:52:50.803495 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.803473    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62b29a26-2b25-4eaa-965e-5f09ed467f11/thanos-sidecar/0.log"
Sep 09 12:52:50.826549 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.826527    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62b29a26-2b25-4eaa-965e-5f09ed467f11/kube-rbac-proxy-web/0.log"
Sep 09 12:52:50.853044 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.853018    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62b29a26-2b25-4eaa-965e-5f09ed467f11/kube-rbac-proxy/0.log"
Sep 09 12:52:50.875848 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.875820    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62b29a26-2b25-4eaa-965e-5f09ed467f11/kube-rbac-proxy-thanos/0.log"
Sep 09 12:52:50.898906 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.898884    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62b29a26-2b25-4eaa-965e-5f09ed467f11/init-config-reloader/0.log"
Sep 09 12:52:50.981673 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:50.981642    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-hbthr_2c573e19-94a2-4af0-8f22-e25fe096f120/prometheus-operator-admission-webhook/0.log"
Sep 09 12:52:51.022674 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:51.022599    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dfc4d978c-58tq2_f45db69e-4fd1-478e-9243-aebfc9a51c50/telemeter-client/0.log"
Sep 09 12:52:51.049282 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:51.049255    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dfc4d978c-58tq2_f45db69e-4fd1-478e-9243-aebfc9a51c50/reload/0.log"
Sep 09 12:52:51.076852 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:51.076822    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dfc4d978c-58tq2_f45db69e-4fd1-478e-9243-aebfc9a51c50/kube-rbac-proxy/0.log"
Sep 09 12:52:53.012692 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:53.012663    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-pc62x_751eecc0-b1e9-46bc-85c2-b18006860c4c/networking-console-plugin/0.log"
Sep 09 12:52:55.025769 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.025730    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"]
Sep 09 12:52:55.032135 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.032111    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.035752 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.035724    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ffhw\"/\"kube-root-ca.crt\""
Sep 09 12:52:55.035752 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.035744    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ffhw\"/\"openshift-service-ca.crt\""
Sep 09 12:52:55.036073 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.036023    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5ffhw\"/\"default-dockercfg-xxfxj\""
Sep 09 12:52:55.038991 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.038932    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"]
Sep 09 12:52:55.136835 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.136792    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-proc\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.137039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.136844    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nmmv\" (UniqueName: \"kubernetes.io/projected/17b44854-9107-42b8-8a8b-2af7d717f7f6-kube-api-access-5nmmv\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.137039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.136975    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-lib-modules\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.137039 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.137012    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-sys\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.137177 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.137084    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-podres\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.238216 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.238178    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-podres\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.238398 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.238237    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-proc\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.238398 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.238275    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nmmv\" (UniqueName: \"kubernetes.io/projected/17b44854-9107-42b8-8a8b-2af7d717f7f6-kube-api-access-5nmmv\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.238398 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.238368    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-podres\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.238398 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.238377    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-lib-modules\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.238613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.238403    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-proc\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.238613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.238425    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-sys\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.238613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.238469    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-lib-modules\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.238613 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.238473    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17b44854-9107-42b8-8a8b-2af7d717f7f6-sys\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.247736 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.247714    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nmmv\" (UniqueName: \"kubernetes.io/projected/17b44854-9107-42b8-8a8b-2af7d717f7f6-kube-api-access-5nmmv\") pod \"perf-node-gather-daemonset-46ft9\" (UID: \"17b44854-9107-42b8-8a8b-2af7d717f7f6\") " pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.343977 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.343901    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:55.613191 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.613111    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"]
Sep 09 12:52:55.616222 ip-10-0-131-207 kubenswrapper[2585]: W0909 12:52:55.616193    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod17b44854_9107_42b8_8a8b_2af7d717f7f6.slice/crio-ad8a0a0bfbe3afa2e578c3ee2f377baa3237fcb61186b185e4238a1c672e4c6c WatchSource:0}: Error finding container ad8a0a0bfbe3afa2e578c3ee2f377baa3237fcb61186b185e4238a1c672e4c6c: Status 404 returned error can't find the container with id ad8a0a0bfbe3afa2e578c3ee2f377baa3237fcb61186b185e4238a1c672e4c6c
Sep 09 12:52:55.735739 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:55.735681    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9" event={"ID":"17b44854-9107-42b8-8a8b-2af7d717f7f6","Type":"ContainerStarted","Data":"ad8a0a0bfbe3afa2e578c3ee2f377baa3237fcb61186b185e4238a1c672e4c6c"}
Sep 09 12:52:56.443795 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:56.443762    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vrrhx_46998610-68a0-4431-bc25-313e17abd0ad/dns/0.log"
Sep 09 12:52:56.468863 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:56.468835    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vrrhx_46998610-68a0-4431-bc25-313e17abd0ad/kube-rbac-proxy/0.log"
Sep 09 12:52:56.514144 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:56.514105    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7d845_72f2bf19-006c-41ff-9f3e-e41c1d113274/dns-node-resolver/0.log"
Sep 09 12:52:56.742661 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:56.742579    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9" event={"ID":"17b44854-9107-42b8-8a8b-2af7d717f7f6","Type":"ContainerStarted","Data":"6a4482ed6c0defcb9735f1873f3de2c5338e69c4a205d50ad14343319246a1fc"}
Sep 09 12:52:56.742822 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:56.742735    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:52:56.759487 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:56.759438    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9" podStartSLOduration=1.759421599 podStartE2EDuration="1.759421599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:52:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:52:56.757456416 +0000 UTC m=+2521.399075620" watchObservedRunningTime="2026-09-09 12:52:56.759421599 +0000 UTC m=+2521.401040855"
Sep 09 12:52:57.219296 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:57.219248    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jwlkw_af64da4a-c9ad-4944-bef3-583521578d0e/node-ca/0.log"
Sep 09 12:52:58.204841 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:58.204796    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp_d9279200-2465-4063-ae1f-59d2116f018a/istio-proxy/0.log"
Sep 09 12:52:58.900734 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:58.900684    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-chp7l_7b6e2ff7-9e23-49c1-aa14-b29047aeef39/payload-processing/0.log"
Sep 09 12:52:59.909006 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:52:59.908974    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gdzzk_1efd99b8-2516-4b3f-8f2f-39cfa33c9c58/serve-healthcheck-canary/0.log"
Sep 09 12:53:00.443652 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:00.443624    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8vfpn_095003f4-af63-4eda-82af-671ce6a44404/kube-rbac-proxy/0.log"
Sep 09 12:53:00.464135 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:00.464110    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8vfpn_095003f4-af63-4eda-82af-671ce6a44404/exporter/0.log"
Sep 09 12:53:00.485161 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:00.485136    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8vfpn_095003f4-af63-4eda-82af-671ce6a44404/extractor/0.log"
Sep 09 12:53:02.757915 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:02.757885    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5ffhw/perf-node-gather-daemonset-46ft9"
Sep 09 12:53:02.759986 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:02.759966    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-92v57_325079e6-e247-4adc-b0e9-a37fae9d7b57/manager/0.log"
Sep 09 12:53:03.678252 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:03.678223    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk_1de04984-4073-43bc-bd71-33e8156d5527/manager/0.log"
Sep 09 12:53:03.701030 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:03.701003    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b9bf5b7c6-5m6xk_1de04984-4073-43bc-bd71-33e8156d5527/copy-manifests/0.log"
Sep 09 12:53:04.346031 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:04.345987    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp_d9279200-2465-4063-ae1f-59d2116f018a/istio-proxy/0.log"
Sep 09 12:53:04.988422 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:04.988395    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-chp7l_7b6e2ff7-9e23-49c1-aa14-b29047aeef39/payload-processing/0.log"
Sep 09 12:53:05.256505 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:05.256482    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-df65498f9-sgpss_8902ddbd-b2ae-4dc7-b360-1664f7d2203f/manager/0.log"
Sep 09 12:53:05.329654 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:05.329620    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-fpgd6_5f0e857b-2645-4746-8d73-e0b10d799af9/openshift-lws-operator/0.log"
Sep 09 12:53:06.563483 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:06.563453    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6cbcdcfcccmvrvp_d9279200-2465-4063-ae1f-59d2116f018a/istio-proxy/0.log"
Sep 09 12:53:07.221313 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:07.221285    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-chp7l_7b6e2ff7-9e23-49c1-aa14-b29047aeef39/payload-processing/0.log"
Sep 09 12:53:11.951157 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:11.951126    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wc5n_5a64c743-7066-4939-9f91-88da5ad84d37/kube-multus-additional-cni-plugins/0.log"
Sep 09 12:53:11.973467 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:11.973435    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wc5n_5a64c743-7066-4939-9f91-88da5ad84d37/egress-router-binary-copy/0.log"
Sep 09 12:53:11.996425 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:11.996392    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wc5n_5a64c743-7066-4939-9f91-88da5ad84d37/cni-plugins/0.log"
Sep 09 12:53:12.020521 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:12.020497    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wc5n_5a64c743-7066-4939-9f91-88da5ad84d37/bond-cni-plugin/0.log"
Sep 09 12:53:12.045284 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:12.045258    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wc5n_5a64c743-7066-4939-9f91-88da5ad84d37/routeoverride-cni/0.log"
Sep 09 12:53:12.067927 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:12.067902    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wc5n_5a64c743-7066-4939-9f91-88da5ad84d37/whereabouts-cni-bincopy/0.log"
Sep 09 12:53:12.089958 ip-10-0-131-207 kubenswrapper[2585]: I0909 12:53:12.089932    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wc5n_5a64c743-7066-4939-9f91-88da5ad84d37/whereabouts-cni/0.log"